Many philosophers believe that a deflationist theory of truth must conservatively extend any base theory to which it is added (roughly: talking about truth shouldn't allow us to establish any new claims about subject-matters not involving truth). But when applied to arithmetic, it's argued, the imposition of a conservativeness requirement leads to a serious objection to deflationism: for the Gödel sentence for Peano Arithmetic (PA) is not a theorem of PA, but becomes one when PA is extended by adding plausible principles governing truth. This paper argues that no such objection succeeds. The issue turns on how we understand the notion of logical consequence implicit in any conservativeness requirement, and whether we possess a categorical conception of the natural numbers (i.e. whether we can rule out so-called "non-standard models"). I offer a disjunctive response: if we possess a categorical conception of arithmetic, then deflationists have principled reason to accept a rich notion of logical consequence according to which the Gödel sentence follows from PA. But if we do not, then the reasons for requiring the derivation of the Gödel sentence lapse, and deflationists are free to accept a conservativeness requirement stated proof-theoretically. Either way, deflationism is in the clear.