You can summon your Retainers from any City-State, but if you want them to sell items in Gridania, then you have to go to Gridania and speak with the Retainer Consignor.