Indeed: the ordinary use of the term 'critique' ('criticism') means to evaluate something. So, a film critic doesn't just tell us how bad a film is, but also how good -- and thus whether certain types of viewers might wish to see it. The philosophical use of the term to analyse something so as to determine its grounds, implications or merit. Thus, a classic type of essay or examination question at University philosophy departments is to 'critically analyse' some idea or argument.
Kant's use is slightly different. A critique of pure reason, of practical reason, or of judgement is not a discussion of an idea or argument, so much as of a whole 'faculty' or 'ability' of the human mind. The three faculties I just listed come from the titles of Kant's three chief critical works, but arguably at least Kant should be understood as also offering critiques of many other 'faculties' such as imagination, understanding, sensibility, or will. In effect, by a 'critique of pure reason', Kant is asking 'what is pure reason good for? '. This involves not only determining what pure reason is (what are its principles, what are its modes of operation) but also how it relates to other mental faculties or achievements. For example, one of the big questions Kant is pursuing is 'Can pure reason, on its own, attain knowledge?'. Critique means to determine, on the basis of principles, the limits of reason -- what reason can and cannot do.