Usually, yes—many card issuers will refund an annual fee if you close the account and request a refund quickly enough. You usually have about 30 days after an annual fee is incurred—sometimes more, sometimes less. It varies highly by issuer and is not always guaranteed. Only your card issuer and agreement with it will be able to tell you for sure.
Credit cardholders should periodically go through their portfolio of credit cards and decide which ones to keep and, perhaps more importantly, which ones to cancel. The card with the handsome welcome bonus might not be so appealing when the annual fee comes due. Because card issuers often change benefits, cards may become less useful or no longer relevant to your spending patterns in terms of rewards offered.