Not only you have to hold the counter button but also go direction away from the attacker.
Once you get a special upgrade you can takedown the guy with a knife instantly if you counter him right. You do the same stuff as with normal dodge but every time the enemy pauses, you, too, let go of the counter button for a moment and then press and hold for the next swing. With normal thugs you have to do it 3 times, with ninjas/assassins 4 times.
I never found that move hard at all...
But then again I'm using a PS3 controller.