At what point does a development company need a dedicated sysadmin?
772
When a company's primary focus is developing, deploying and maintaining software, at what point would it be prudent to hire a dedicated sysadmin? What are the warning signs that you need one?
The person or persons who are handling system administration tasks are spending a significant percentage of their time on those tasks. The percentage may vary from organization to organization, but it's a tradeoff on that person spending time on system administration versus their primary responsibility. Another factor is the quality of the network service and up time. If you have someone who is barely managing the network or is having trouble keeping it up, then you should have a dedicated person who knows the network and can act in a proactive manner.
So some basic rules of thumb:
Do you have outages or degraded network service that could be handled by an experienced system administrator?
Are the current people handling system admins utilizing more than 30% of their time on system admin tasks (the % may vary).
As a contractor who provides sysadmin services to small companies, including ASIC and hardware design companies, I have this perspective:
A sysadmin's value is not primarily the difference between his direct cost and the direct cost of engineers who are sysadmin'ing rather than engineer'ing.
In my experience, most (but not all) engineers are good engineers but lousy sysadmins. They know how to do the engineering side, but building networks, servers, and services are not what they do all the time.
A sysadmin can increase his value if he can improve the productivity of the engineers -- for one particular industry I work in, we doubled or more the units of work done through a regression and simulation server farm by changing their networking infrastructure, increasing the value of their server farm, engineers, and very expensive tools licenses. This was a far greater value to the business than merely freeing the engineers from sysadmin duties.
Of course, not every company has low-hanging fruit like that!..
Value is also increased by avoiding downtimes that prevent larger numbers of people from working. I always say, if your IT resource never has to be on-site, then he's doing a pretty good job...
What you need is an evaluation from someone you trust, not just someone looking for a job or looking to sell you something (although they will be doing that, too). Someone who will come in and look at your site, front to back, and ask you questions about what you do, how you do it, and what you are trying to accomplish -- as these are all very different things!
Finding these people can be easy or hard, depending :) I suggest that you talk to your network of peers in other industries and ask them who they have used in the past, good or bad. Do your research. Ask for, and check, references.
What you also need to do is recognize that this will cost money. Time for a FT/P or a consultant (or team of consultants). I personally prefer customers to engage my company with a small fixed number of on-site hours per week. They then have access to my whole team and their varied expertise so that our responses can scale up (or out) when required without them having to have a large FT/P staff constantly on the payroll. At some point it makes sense to go with a FT/P of your own, if only as a senior IT or IT manager. But you don't need to go there right away if you get the right team.
Also realize that any new body will want to change the way some things are done, and initial changes can be bumpy. This is why you have to trust who you get to take you from where you are to some tradeoff between where you want to be and what you can/are willing to afford.
If you are asking the question, then the time is right to start doing research, and maybe have one or two consultants in to do an evaluation. Then take what they say and decide what the next step is.
When the amount of productivity you're losing by having developers and admin staff filling in for the absent sysadmin is greater than or equal to the sysadmin's salary.
Don't wait until your developers are getting bogged down with sysadmin-type tasks. Just about every developer I've met has some capacity to satisfy some level of system administration without giving any visible indication that they are becoming consumed. By the time you notice that your development team is being visibly impacted by the sysadmin load, you've probably already spent more of their time on it than you would care to.
Conversely, early on your system administrator is likely to be underutilized in the sysadmin role. Consider integrating them into the development team. I've never seen a software product suffer because it had the early input of an actively participating IT resource contributing feedback on technical requirements. It's usually the case that IT considerations get short shrift without that resource being involved.
If you're asking the question, my guess is that you've already observed behavior that indicates the tipping point has been reached.
I would recommend creating a log for a month. Every time you or an employee spends time on a sysadmin task, log who it was and how much time it took. At the end of the month calculate how much money it cost total for all that time, and double it. By doubling it you represent the amount of time you paid to have those people do sys admin tasks AND amount of time lost that could have been used to generate revenue (i.e. coding).
Once you have that number, ask yourself "Can I get a full time sys admin for that price per month?"
If yes... hire someone.
Also, look at service companies in your area. There are many that work on a retainer situation. This would be a good solution for that period between when you need someone but not full time. Know that a full time system admin is not your only option. Also, you don't have to deal with benefits and other personnel issues.
NOTE: I work for a company that provides these services to small business so I may be a bit biased.
On a more serious note, you should when your developers are spending the time they could be developing on administration and operations. These tend to be very different skill sets so it is good to have someone come on board (even on a part time basis) ahead of the crunch so that you avoid situations like the one above.
It's also worth considering that a Sysadmin will bring experiences and knowledge that most developers won't have. They'll know how networks work, how to get the most out of databases, how to make applications scale. They tend to have a slightly different mindset to most developers, and that knowledge can be invaluable.
Doesn't matter all the 'if' clauses. On average, you'll find a 5:1 in emerging companies.
there are many metrics out there, such as "30% of developer time spend doing sysadmin work".
My personal (18 years) experience is that, by the time you have 5 folks developing code, someone needs to come in to help build centralized systems/services so those 5 good folks can continue being productive at writing code and not managing servers/software systems, and provide return on the (salary) investment.
The person or persons who are handling system administration tasks are spending a significant percentage of their time on those tasks. The percentage may vary from organization to organization, but it's a tradeoff on that person spending time on system administration versus their primary responsibility. Another factor is the quality of the network service and up time. If you have someone who is barely managing the network or is having trouble keeping it up, then you should have a dedicated person who knows the network and can act in a proactive manner.
So some basic rules of thumb:
As a contractor who provides sysadmin services to small companies, including ASIC and hardware design companies, I have this perspective:
A sysadmin's value is not primarily the difference between his direct cost and the direct cost of engineers who are sysadmin'ing rather than engineer'ing.
In my experience, most (but not all) engineers are good engineers but lousy sysadmins. They know how to do the engineering side, but building networks, servers, and services are not what they do all the time.
A sysadmin can increase his value if he can improve the productivity of the engineers -- for one particular industry I work in, we doubled or more the units of work done through a regression and simulation server farm by changing their networking infrastructure, increasing the value of their server farm, engineers, and very expensive tools licenses. This was a far greater value to the business than merely freeing the engineers from sysadmin duties.
Of course, not every company has low-hanging fruit like that!..
Value is also increased by avoiding downtimes that prevent larger numbers of people from working. I always say, if your IT resource never has to be on-site, then he's doing a pretty good job...
What you need is an evaluation from someone you trust, not just someone looking for a job or looking to sell you something (although they will be doing that, too). Someone who will come in and look at your site, front to back, and ask you questions about what you do, how you do it, and what you are trying to accomplish -- as these are all very different things!
Finding these people can be easy or hard, depending :) I suggest that you talk to your network of peers in other industries and ask them who they have used in the past, good or bad. Do your research. Ask for, and check, references.
What you also need to do is recognize that this will cost money. Time for a FT/P or a consultant (or team of consultants). I personally prefer customers to engage my company with a small fixed number of on-site hours per week. They then have access to my whole team and their varied expertise so that our responses can scale up (or out) when required without them having to have a large FT/P staff constantly on the payroll. At some point it makes sense to go with a FT/P of your own, if only as a senior IT or IT manager. But you don't need to go there right away if you get the right team.
Also realize that any new body will want to change the way some things are done, and initial changes can be bumpy. This is why you have to trust who you get to take you from where you are to some tradeoff between where you want to be and what you can/are willing to afford.
If you are asking the question, then the time is right to start doing research, and maybe have one or two consultants in to do an evaluation. Then take what they say and decide what the next step is.
Probably just before something bad happends:
You realize you dont have a regular backup of a Line of Business app - such as something tracking licenses/customers
You can't remember when production servers were patched or scheduled for maintenance
You discover your file server is dead becuase TWO drives failed in your RAID array
It takes longer than 1 day to get a new machine provisioned to replace of an old/existing machine
Printers are out of toner or say "maintnenace required" on the LCD readout
"All your base are belong to us" or similar network intrusion
E-mail server is overwhelmed by the latest spam attack
I am sure there are other WTF moments that I havent listed in my short list.
When the amount of productivity you're losing by having developers and admin staff filling in for the absent sysadmin is greater than or equal to the sysadmin's salary.
Don't wait until your developers are getting bogged down with sysadmin-type tasks. Just about every developer I've met has some capacity to satisfy some level of system administration without giving any visible indication that they are becoming consumed. By the time you notice that your development team is being visibly impacted by the sysadmin load, you've probably already spent more of their time on it than you would care to.
Conversely, early on your system administrator is likely to be underutilized in the sysadmin role. Consider integrating them into the development team. I've never seen a software product suffer because it had the early input of an actively participating IT resource contributing feedback on technical requirements. It's usually the case that IT considerations get short shrift without that resource being involved.
If you're asking the question, my guess is that you've already observed behavior that indicates the tipping point has been reached.
I would recommend creating a log for a month. Every time you or an employee spends time on a sysadmin task, log who it was and how much time it took. At the end of the month calculate how much money it cost total for all that time, and double it. By doubling it you represent the amount of time you paid to have those people do sys admin tasks AND amount of time lost that could have been used to generate revenue (i.e. coding).
Once you have that number, ask yourself "Can I get a full time sys admin for that price per month?"
If yes... hire someone.
Also, look at service companies in your area. There are many that work on a retainer situation. This would be a good solution for that period between when you need someone but not full time. Know that a full time system admin is not your only option. Also, you don't have to deal with benefits and other personnel issues. NOTE: I work for a company that provides these services to small business so I may be a bit biased.
When your computer room looks like this. =)
(source: tvsnob.com)
On a more serious note, you should when your developers are spending the time they could be developing on administration and operations. These tend to be very different skill sets so it is good to have someone come on board (even on a part time basis) ahead of the crunch so that you avoid situations like the one above.
It's also worth considering that a Sysadmin will bring experiences and knowledge that most developers won't have. They'll know how networks work, how to get the most out of databases, how to make applications scale. They tend to have a slightly different mindset to most developers, and that knowledge can be invaluable.
Depending on how big you are, I would start looking when
Simple: 5 to 1.
Doesn't matter all the 'if' clauses. On average, you'll find a 5:1 in emerging companies. there are many metrics out there, such as "30% of developer time spend doing sysadmin work".
My personal (18 years) experience is that, by the time you have 5 folks developing code, someone needs to come in to help build centralized systems/services so those 5 good folks can continue being productive at writing code and not managing servers/software systems, and provide return on the (salary) investment.