Before you can succeed in creating your positioning, messaging, branding, and marketing execution, you need to establish your core identity. What your best? What can only you do? For who?
To answer the questions “What’s it for?” and “Who’s it for?” I often start with this model to look at the complete playing field of competition and yourself to determine where there is a need in the market. You basically list your competitors and plot them by price positioning and completeness of the solution. The four quadrants are basically Subaru (niche), Lexus (premium), Toyota (value) and Daihatsu (volume, white-label, etc). This helps find potential niches that are underserved.
If you find it's too crowded, you can start applying scope based on your own strength. You could base this on your existing customer track record and do this exercise for various verticals. If you find one where the premium- or value quadrant is empty, and it's a growing market, you can strike gold. For example, being the premium software house for medical device manufacturers (growing, they need software, and will pay a premium for deep expertise, unique experience release related to regulatory needs).
Also, what do your customers tell you makes you unique? Does this translate to a specific market segment? (Working with large or small clients? Vertical segments? Types of solutions?).
Finally, I really like Michael Schrage's book. It's always helped me focus on a very specific mission for the customer that then helps focus on a niche where that's the best fit.
In the end, it usually comes down to a combo of segmentation choices:
Pick in an industry that you can relate to (based on existing/past clients)
Ideally, find customers that are small enough to learn from, and big enough to pay you, and become meaningful references
Work with roles who have decision making power so you don't waste your time (BANT)
Is the “Job-to-be-Done” aligned with your strengths and track record? Is it aligned with KPIs you can help deliver against?
Is your target audience open to the stage of maturity that your category warrants (I recommend reading Geoffrey Moore’s “Crossing the chasm”)
The most powerful words to find your niche to nail are 'best' and 'only'. When you find something that 'only you can do' or that you're 'best at' in a specific part of the market, you should explore that segment of the market. If it's growing and fun to work in, you might have found your niche. Sometimes doing the feature matrix product planning work is a good way to get going if you’re stuck and don’t know what makes you really unique.