Are your IT users stuck in the past, and do your nerds know how to handle them?

Is this clip early familiar. Do your users often have repeated stupid problems. Do your IT people get frustrated and yell at them? Has HR ever had to step in between IT and your users? If so It may be time to fire your IT guys (and maybe more).

Working for an IT consulting company has its upsides. Our internal users inherently know and have a passion for technology, and as a result we internally do not have to field stupid questions about how to turn on a PC (However our users do ask for support with SSL-VPN to their virtual desktop running on their iPAD so it is a trade off).

In most organizations 10% of the users are responsible for 90% of our help desk’s issues. Learning how to gracefully deal with these users, and their problems has made our managed service guys endeared to users, and as a result we do not have angry women showing up to beat up our nerds.

The first step is identifying the baseline IT intelligence of an organization and identifying if the problem is a quick fix, a short educational opportunity, or a larger scale process change. In a rare case, the problem is the user and after an internal escalation and sanity check a call is place to their management to deal with this user with their own HR resources.

In some cases where the user will likely never have to touch whatever is the problem again once fixed, simply logging into the users computer and making a quick change without explaining anything is the best practice. An example is adding a signature in Outlook. Once applied there is rarely a need for any changes to be made.

In other cases, we can identify that this is something the user should be capable of handling on their own and we explain how they can make the change themselves. An example of this would be walking a user through how to Connect and disconnect from VPN.

When we recognize that a problem is a persistent one and the solution is above the baseline knowledge we may sugest purchase of additional software, a larger change in the organization to fundamentally remove the opportunity for the problem. An example of this is removing the problems of using VPN to get to email, by deploying Exchange Webmail and Activesync and removing the requirement for VPN to be used to access mail on phones and from home.

In the rare case where a user makes an outrageous demand (My VPN doesn’t work from rural china over a dial up modem running over string and tin cans!), or is burning large amounts of internal and external resources (like my time) we will have a chat with management and suggest they limit the user’s demands on IT, or perhaps hire a replacement who can turn on their computer.  Anyone have any cases where they feel they need to start firing users?

Posted in Uncategorized | Leave a comment

Do Your IT guys Exist?

Does your It guy have too much in common with Santa Claus?  I’m not talking about that strange guy who has a spare tire and a beard who works on your phone system and mutters under his breath about how everything should have a command line.

Creepy phone guy should be fired too but that's another post...

Do your IT guys mysteriously appear and then quickly disappear at different times of the year?  Do they only show up with presents (or solutions) in the months of their choosing?  Do you letters and requests fall on bounced email addresses and filled voicemail?  If so you should fire your IT guys!

There are a number of signs that your IT guys do not actually exist and in making this post more “real” I’ve decided to use Houston based Iron Tech Solutions as an actual example of how to spot the key signs.

The first key sign that they are as mythical as the tooth fairy is if their website not only goes down but stays down for months.  This site in particular hasn’t been seen online by the Internet Wayback Machine since 2009, but they keep on trying to sign up new customers.

October is crunch time for toy making, wait a couple months....

The second step is the one way communication punctuated by only annual (or semi-anual if you practice Christmas in July) responses to clients.  To their credit this guy’s complaint was filed at 9:57PM so he clearly should have been in bed dreaming of sugar plum fairies rather than begging for a functional FTP server.

F is is for freaking awsome!

But we got all A's in History...

The last step in identifying if your IT is a “real boy” or belongs on the “island of misfit toys” is how they are viewed by the rest of the good boys and girls.  If the only proof of your existence is being on someone’s Naughty list its time to fire your IT guys.  For IT companies looking for help on actually being “real” online, I’d direct people to Red fish Blue Fish Media.  They do great work across all levels (website/social media/email campaigns/SCO).

Posted in IT service Company | 2 Comments

Datacenter? We don’t need no stinking Datacenter!

Here at FYIG, we like Datacenters. Not just because we can bask in flourescent glory while hacking away at our favorite passtime (though that’s nice). Not even because we can work without fear of being asked whether the toner light on the printer means you should replace the toner cartridge. No, the reason we like datacenters, is because datacenter = vacation. The levels of reliability we can achieve actually afford us the opportunity to take a vacation without fear of what might happen in our absense.

Good datacenter facilities can be somewhat expensive however, and we often find that IT guys will go to great lengths to avoid having to either ask for more budget, or deny themselves the year’s server purchases. All too often we find business critical systems running in a broom closet, in the basement, or under an IT guy’s desk.

This is *not* where you want to find mission critical hardware

Ask yourself this: If you were to face an extended (>1 week) power outage at your primary office, where would you rather be? 1,000 miles away from ground zero sipping drinks with little toothpick umbrellas in them while surfing your E-Mail content with the knowledge that your systems are fine? Or running along the freeway brandishing a rack rail waving it in fury at FEMA trucks driving by in an attempt to re-hijack your generator delivery?

For those of you who chose option #2, we have a load of old server rails we will donate to your cause. For the rest of us, I think the need for reliability is clear.

Posted in Uncategorized | 2 Comments

Do you really need more speed in your servers?

So I hear Honda's are really reliable...

I’m going to come out and admit that love Honda cars.  They get great gas milage, they are afordable, and best of all they are super reliable. Some people  want speed at the cost of all else.  Reliability, safety and storage are the last thing on their minds, as they get a high from just looking at something fast.  There is a time and place for both attitudes, but it is not in your server room.

When you’re looking at upgrades, is your IT guy obsessing over Ghz?  Is he recommending a 1:1 replacement of 5 year old servers with new ones that run circles around the old ones? Any server upgrade should be made for strategic reasons, and while old servers do need to be replaced for efficiency/reliability and a host of other reasons, the FYIG staff finds that its uncommon to find a workload that a couple dozen old servers used to handle that can not be run on three new beefy machines thanks to the magical pixie dust called virtualization.  Given big enough servers, you can have one fail and the other two will take over for their fallen comrade seamlessly. (There will be a post about this).   If your IT guys are wanting more than 3 servers, you should be wanting some serious performance metrics out of them explaining why this is the case, and evidence that your utilization graph will not look like the above graph when you’re done.

Often times IT guys will shave off support (1 year instead of 3 or 5) so they can get more money for speed.  This is like buying a new air intake, and not fixing your brakes.  It will get you the scene of the crash a lot faster, but I’m not sure if thats where you or your business want to be.  Another thing that is often left out is software licensing. (True outlaw drag racers don’t concern themselves with th law, they are too fast to get caught!).  Even simple options like out of band management, and monitoring (being able to access and turn a server on and off remotely) are useless to the speed freak in your server room, as unified and simplified management might make him unnecessary. Kinda like my Mechanic after I traded in my exotic sports car for a Honda.

Posted in Uncategorized | 3 Comments

The lights are on, but no one is home…

Server rooms are often reminiscent of a carnival. Lots of flashing lights in an assortment of pretty colors. Oh, and lots of scruffy guys with greying beards mumbling under their breath while they shuffle about looking important, but I digress…

While this can be quite the sight, not all of these flashing lights bring about joy and happiness. Take this one for example:

If the IT guys (or gals) aren't running to fix this, Fire them!

That pretty looking orange light is an indication that the drive has failed. I often take it for granted that everyone would realize that the single orange light among a sea of green might just indicate a problem, but we at FYIG run into this all too often. One drive fails, then another, then another. The IT guys involved either do not realize that this means badness, do not care, or (most recently) attempt to ignore the issue altogether. In one recent incident, pointing out the drive failure resulted in a frenzied attempt to change the subject and pretend like nothing happened.

Why is this bad though? This is why I have RAID right? Not exactly. While RAID does protect you from immediate badness in the case of a failed hard drive, it is intended only as a temporary solution. There are a number of reasons for this, but the most pervasive is the simple fact that hard drives fail in groups. The vast majority of servers (and even many high end storage arrays) utilize drives that were manufactured at nearly the same time (in some cases, we have even observed sequential serial numbers). Essentially this means that any small defects or variations in the drive’s construction will affect all of the drives in a particular device at nearly the same time. In practice, this means that a dead drive is the surest indicator that another drive is going to fail in short order.

Couple this with the proliferation of RAID 5 in recent years, and the situation becomes even worse. Not only does a RAID 5 array suffer a severe performance hit while operating with a failed drive; but it also operates at considerably higher load, increasing the odds further of a subsequent failure.

Side note: Don’t use RAID5. Details in another post.

Ultimately, hard drives are cheap. Really cheap. Even if you’re paying a SAN vendor’s drive tax, they are *still* really cheap, compared to the cost of a rebuild from backup.

So when you see the flashy orange drive light, please, shell out the $300 and replace the darn thing.

Posted in Uncategorized | Leave a comment

Does your server rack look like this?

Does your server rack have something like this in it? Are the nerds explaining how its the next big thing in computing.  Has it been mostly empty for a while? You’ve been had, your money was wasted, and you should fire your IT guys who recommended it to you.   Welcome to a blog devoted to outing bad IT practices that are costing you money, time, performance objectives.

2 Blades 14 more to go!

2 Blades 14 more to go!

This is a Blade Server Chassis (Dell in particular) and this is an example of one with 2 servers installed.  The idea is that you buy stripped down servers that share power supplies, network and storage switches, as well as large fans allowing for super dense deployment of servers with far fewer cables and a single vendor that supports the entire thing.  In some cases it can be a good idea, you spend some money up front and once you fill it up you save on cabling, hassle and redundant components. When you see one like this however (especially one that is two years old or more) you’ve been had. The chassis itself is a box with just fans and power supplies runs a minimum of $30k and after support/networking/storage network and other goodies are added this easily doubles. This is fine if its supporting all 16 half hight servers and reducing cost from fully loading those, but in a case like this your paying two to three times what servers costs, and by the time you get around to filling it out 5 years later the chassis itself will be out of date, causing an expensive rip and replace of not just your servers but ALL of your infrastructure to start this over. I’ve seen one of these loaded with only the equivalent of 3 desktops sold installed and configured for $150,000.

Why do people buy them then abandon them?

  • Unrealistic growth and budget forecasting
  • Budget forecasting that comes in big chucks every 4-5 years
  • They Heard it was the “Enterprise” way of doing this
  • Complete lack of understanding of performance needs to accomplish business objectives
  • Find an excuse to put it on another department’s budget
The FYIG’s take:
If blades are being deployed they should be deployed at least two thirds with plans for being full within a year. Your IT staff should have clear tracking of CPU/Memory/Storage/Networking usage metric’s and how this dense expensive rapid growth tool is going to solve current problems as well as the clearly projected near future.

About FYIG:
Fire Your It Guys is a blog devoted to calling out bad IT, and specifically calling out the individuals that implement it. This could be anyone from the lonely desktop tech who creates spaghetti wiring, to the CIO who deploys $100,000 gadgets that functionally do nothing but blink lights at you.  Our team hopes to make this an interactive blog and we will be initially taking email’s and working towards this becoming a user driven website.
Please feel free to send us questions and quick pictures! The format will range from pictures and explanations like this article to short tweets of IT behaving badly.  As IT is projected to continue to loose over 100,000 jobs a year, we would like to start a discussion here between the insiders, the C-suite and the grunts about what are simple indicators to tell the difference between the cream of the crop, and the flops.
– The Consultant
Posted in Blade Server | Tagged | 4 Comments