a one-horse town

A one-horse town originated as a literal term for describing a small town, hence a one-horse town. Small towns in history often had just one horse throughout the entire town, typically an unimportant place in a rural area. In modern usage of the word, when talking about a one-horse town, they are speaking of an unknown, small and secluded town most people dont know about.