You have to take the lowest roll when you have disadvantage (as mentioned by DaFluid). So you have a critical hit only if you roll two 20, which is rather unlikely, obviously.
You may calculate the likelihood: since you have 5% to roll a 20 (1 / 20 = 0.05), you have 0.05 * 0.05 = 0.0025, or 0.25% of rolling two 20 when rolling an attack with disadvantage.
Note that there are some classes and feats that give you the possibility to get a 19 or a 20 and have a critical hit. In that case, you increase your chances by ×4 since you may get any of (20, 20), (19, 20), (20, 19), or (19, 19) to get a critical hit. So 1% chance instead of 0.25%.