This is a constantly recurring question that comes up in role definition, trying to set developer levels, job titles and overall in HR for developers. And most answers come down to a list of metrics that must be met for each level. Number of years, experience with the business, amount of people they can manage, and so on. In a really structured and bureaucratic environment, maybe you have no choice but to set these as strict values and levels.
In our startups we don’t do this. Instead we ask one main question to the interviewing team when hiring: “How long would you trust this developer to code before wanting to see what they are up to?”
And that is the most accurate form of instantly knowing the developer’s level. Someone you check in with twice a day is obviously junior, a few days intermediate, a week higher intermediate, a few weeks of runway is more senior, over a month is very senior.
We then split the more senior people into two categories: “heads down do-my-own-work and rock it” and the other type is “their presence makes the whole team stronger.” We have need for both types, and not all senior people should or want to help or mentor others. So for the latter group who can also mentor, we make it part of their job to reserve time for the rest of the team and avoid overloading them.
We also avoid architect vs. developer job titles. If you have design/architecture responsibility it is a role for a given project and not your job title. You are out front, leading, have a vision, can picture the concept and have it under control – therefore you are acting like an architect for that moment. But there is no need for this title.
As for CTO, this isn’t even in the same category for many companies and involves so much more than just development and architecture experience. Which is why many companies have so many problems with technology, because they drop a senior engineer into this role who brings along their baggage and injects their flaws into the whole company. I wouldn’t include this in the same discussion at all.
Finally, just remember when thinking about developer levels those two questions that provide the core value of the developer and look at the “whole package” of the person. “How long before we want to see what you are up to?” and “do you just do your own work, or do you make the whole team better?”