FSG Director D. Ray Marler resigned from the bank's board last month.
In a regulatory filing, First Security Group said Marler was leaving "to devote additional time to his other business interests and his family," and he didn't have any disagreements with FSG's operations or policies.
No successor has been named, although the bank said it does intend to fill the board seat.
Marler, who owns a construction and waste management business, was paid $13,000 for advice and services for the bank in the past couple of years. Marler also bought some bank repossessed property for $20,000, according to an outside audit of FSG that criticized the lack of independent spending controls at the bank.