Change is coming

I can’t believe its been over a year since I wrote my last post. Life has been crazy with work, family, renovating my house and an old car I have in the shed taking pretty much every waking hour of the last 12 months.

I moved into a new role in the third quarter of last year and not only did I feel like I had been thrown in the deep end, I felt like I had concrete boots on at the same time. It’s strange when we whine about our bosses and the decisions they make but when you get given their job to do you start realising the challenges they faced and reflect on some of the decisions they made you didn’t like and can now see some of their logic and perhaps begin hating yourself a little for it.

So I am now part of a different team and our organisation is focusing on its leadership for the coming year so we can expect many off-site group hug type get togethers where we will work towards becoming a more cohesive team. I am told to prioritise “the team you are in over the team you manage”. Not sure how the team I manage will feel about that but only time will tell I guess.

In the last year many IT strategies/technologies that have been in the private sector for many years have reared their head in government and gained more mainstream support and as such have become part of the whole of government strategic direction. Terms such as DevOps, Automation, AI, Data Science and BlockChain are becoming common place and there is a real push for the government sector to take advantage of these technologies/methodologies  for the betterment of the tax payer and the state as a whole.

The problem is many government departments like ours are overflowing with legacy systems. We get sold a cloud option and the IT department are scrambling around trying to figure out how to migrate our old VB6 or Cold Fusion application across into the electronic ether. Few of the cloud benefits are relevant to our old applications as they don’t load balance the same, we can’t just seamlessly add another application server and expect the application to accept it and we can’t just automatically script it across to a fail over and stand it up without upsetting the masses or by making less than significant configuration changes to get it back into the mix.

On top of that we have been running some of our stuff on Windows 2003 or even Windows NT OS’s and funny enough they don’t offer that in the cloud as PAAS so we are limited to IAAS paying a premium for hardware that can run a 64bit OS while only running a 32bit OS on it. Not very cost effective. The truth is in many scenarios we have been sweating our infrastructure for so long that its out of support and we have accepted that risk in the event of catastrophic failure.

But maybe as a government department we shouldn’t be sweating our stuff for so long and should be protecting the tax payers investment or information with more care and security. Perhaps the cloud will offer us the much needed upgrades for a price that is within our reach. If anything it will certainly push us to upgrade or rewrite some of our old legacy applications and really evaluate and justify their existence within our organisations.

When I first started in IT I used to look at these old applications with a sense of nostalgia. They were rock solid. A decade or more of bug fixes had made them practically error free so we got few calls for defect related issues. Everyone that used it knew it back to front and knew its idiosyncrasies and how to get it to do what they wanted and had by now accepted what it couldn’t do. The older staff would teach the younger staff how to use it and the support contracts on the underlying platform had long since expired but we hadn’t had an outage in years so didn’t need them anyway. I compared these legacy systems to an old car. They didn’t have any bells and whistles but did the job they were built to do and took little effort to keep them going as they were so simple by comparison to today’s systems.

Then tragedy strikes. Someone asked for an enhancement, the underlying application server started to struggle, the database started to have disk write errors and when the time come to actually fix it, it was harder than expected. The application didn’t run on modern hardware, replacement hardware could only sourced on Ebay (NASA did it but apparently we’re not allowed?), none of the outsourced developers had ever coded or even analysed this suite in the past and pulling it to bits now was a real stretch of their analytical abilities. Failing that we have to pay an “older” developer who knows the technology and actually wrote the system 15 years earlier, a bucket of money to come out of retirement to fix it for us, including taxi fare for 1000 km round trip to and from where he retired to (and yes we did actually do this, TWICE).

I start thinking that perhaps the old car analogy was even more appropriate. Nice to look at but can’t be relied upon for day to day operation. New cars are more efficient and those bells and whistles (engine management and anti skid brakes/RESTful API’s and contemporary technology) certainly make life easier and safer. On top of that we start to evaluate the application suite in its entirety and evaluate out business model and if we are doing things the same now as we were 15 years ago then we are surely but slowly getting left behind.

The next few articles I would like to do will be about these new technologies and the challenges we as an organisation will face implementing them and perhaps throw into the mix the odd article around other new technological buzzwords for the day. In fact my next one will be about CryptCurrency and not so much around the technicalities but more about the social aspect of some of the various currencies that are out there and the good (or bad) they can do.

As per my first post this blog is more about me trying to get my thoughts in order than trying to inform anyone. So in that sense its more a journal than a blog. But thanks for the support/critique I have received to date.  All comments are welcome.

Working from home

It’s been awhile since my last post and the reasons are varied but it’s the usual “life gets in the way” of something I want to do. I am a fan of articles around “Work / Life Balance” but to be honest some of them are fantasies and by the end of this you might think the same.

I think the best work life balance you can aspire to is not working at all. The second best though must be working from home. Some of us do this full time and have done for a while. Some of us do this part time or at random intervals throughout our careers. Unfortunately some of us just dream of doing this. Isn’t it the dream to wake up and work in your underwear? Having an infinitely flexible work day, with lots of productivity, little to no distractions and all importantly no travel time. This is the ultimate goal of the remote workplace, zero travel time.

Or course the reality of it is somewhat removed from that for most.

People work from home all of the time as all you realistically need to work in IT (or any other information services role) is an internet connection to do what is needed.

Unfortunately to do it successfully and be productive at home is another thing as there are multiple distractions and other reasons when we do work at home that we don’t always achieve our goals of a fruitful working day. It has taken me the longest time to figure this out and these are some of the lessons I have learnt over my career.

Lesson #1 – Clocking on
I started my working from home career by getting out of bed at about 5am, logging onto my desktop, remoting into a server that sat in a dusty old storeroom attached to a dusty old office in some remote part of Africa that when you flew there took 3 days and 4 flights (where the last leg of the flight saw you with a cage with a chicken in it on your lap) to extract results out of a collection of mining equipment to generate some daily reports. To kick off the Remote Desktop Connection (RDC) to this part of the world took about 5 to 8 minutes so I used to go off and make a cup of tea. I would come back, run some checks and then kick off the extraction process which again would take a few minutes. So again I would head off and get some clothes on. Come back and the files were extracted and waiting and I would run some scripts that I had written to manipulate the data and then run the reports, format them and send them via email to various parties.

Time for breakfast. Did I mention you eat more when you work from home?

So I would break my day into smaller chunks of work and take breaks in between to make a cuppa or a snack, read the news, get the mail, etc, etc. When you’re at home all day there in an abundant amount of things to be done so you do them with the justification that you started work at 5am and will make up the hours as the day goes by. You end up doing little bits of pieces here and there and try and make up a 8 to 10 hour day but it takes you to 10 or 11 at night to do those 8 hours.

This was before I had young kids which would no doubt input an influx of other interruptions throughout the day.

So my advice here is to Time box your working day. Schedule your interruptions (those that can be scheduled) around your work schedule. If you have to get kids ready for school and pick them up then book that into your day. If you like getting up at 5am then schedule to start at 5:30am and give yourself 30 minutes for coffee and breakfast and getting dressed and be ready for work at 5:30. If the kids get up at 6:30 then you know you have an hour or more to get a decent piece of work in before dealing with them.

Plan to stop for lunch (it’s mandatory in some organisations and for good reason). Plan to pick the kids up and if you can’t get a solid hour in between sorting them out then don’t bother. Waiting until they go to bed is sometimes an option if you need to make up the time. If you are like me and you don’t watch much TV then do an hour or so then before relaxing for the night.

With running around with kids and work you have probably done a 10 or 11 hour day so schedule time to put your feet up and turn off.

Remember the key lesson is to timebox your work day and not just work between interruptions.

Lesson #2 – Dress for Success
I mentioned above the dream of working in your underwear but that is perhaps best left for lingerie models and surf lifesavers. I tried it for a while but couldn’t seem to get into work mode while sitting there half naked. It also felt awkward when talking on the phone to clients.

I am now a firm believer of getting dressed for work. It may not be a suit and tie or even business dress but it has to be something different to your pyjamas or your underwear so you are actually in work mode.

In winter to avoid the cost of heating the house all day just for one person I wore my official work uniform which consisted of Ugg boots, an old tracksuit and a beanie. It wasn’t flash but it was comfortable and became my routine through most of winter.

Routine is important when starting the day and puts you into a work mode if you do the same thing every day.

Lesson #3 – Get rid of the distractions
I am the master of procrastination so if I have something within my sight that looks like it even remotely needs my attention and it’s not work related I am all over it. I will reorganise my desk, pay bills, adjust my chair, upgrade some software on my PC, read the online news, reply to a private email, etc, etc. None of this is very productive so I ensure when I finish each afternoon I maintain a clean desk policy so in the morning I sit down and am ready for work with nothing to entice me away from my work.

I make up a quick list before I go so that when I sit down in the morning I am not wondering what to do. I have a list of priority items that needs resolving and it puts me in work mode and I get stuck into them. This gives me a sense of quick achievement and puts me on the productivity path for the remainder of the day.

Know what distracts you, deal with it and if you find it hard to get started in the morning, ready a list before you finish at night to kick start your morning.

Lesson #4 – Be realistic about what you can and can’t do from home
Even as an information worker with complete remote access to all of my resources that I have at work there are still a multitude of items that I cannot do at home.

Meetings should be face to face if possible. Sorry but I am not buying the “digital workplace” with the teleconferencing as a viable option to replace a face to face meetings. So much more can be achieved in person than on the phone or via a video conference.

Workplace relationships are very important to what I do and communicating via email, instant messaging or video link just won’t cut it. Sure, for just sending or receiving some basic information and putting something specific in writing is always an option for a large chunk of the knowledge transfer that takes place on any given day but I need to be in the same room as the person I am talking to if there is a possibility that the issue is a contentious one or if I am asking for a favour.

Lesson #5 – Clocking off
I found when I first starting working from home that I forgot to clock off. You leave your PC still logged in at the end of the day. You go and have dinner, come back and read a few emails. You watch some TV and have a quick look during the ad breaks. Or you take your laptop into the lounge and half watch TV while working. You end up doing bits and pieces between doing other things and end up working until midnight without really being very productive at all. .

You go to bed, wake up in the morning, sit in front of the PC and it feels like you haven’t clocked off yet.

When you commute to and from work, there is this separation that exists. When you work from home you have to create that separation yourself. I have already mentioned about clocking on. That is the moment you are no longer at home but now at work. So you need to ensure you can break away from the work space or out of work mode completely and let yourself reboot. I find that once I break away from work I don’t necessarily stop thinking about work but it helps my brain absorb what has occurred throughout the day and put it into some sort of order.

I often work on my most difficult tasks last if I am having trouble formulating a solution to something. This then sticks in my mind and I go to bed with it and have on many occasion woken at 4am with a solution.

Many of us aim to work from home and it seems like the dream job. The reality is it actually takes significant amounts of discipline to be successful and productive at it. Without formality around working from home you are doomed to fail or flounder all day and be unproductive.

Enhancements During Lean Times

crack-graphic-1236437

 

So our job is to keep the lights on and I have talked about doing the odd minor change to keep the bill payers happy. But what do we do when funding is tight and we have to constrict our funding to do the bare minimum and “REALLY just keep the lights on”? How do we still maintain the illusion of being responsive to the business needs as they will no doubt be under pressure to change the way they do things, and as a result the systems that enable those things will need changing also.

 

So here is the source of the problem.

 

bus del pyramid
 

 

What the business wants
The business wants everything they ask for and they want it for nothing. They want every submission or request for change they put in will be actioned accordingly and presented to them with a bow on top. And why shouldn’t they? After all, the IT department doesn’t make the organisation any money and doesn’t form part of its core business and yet it spends millions of dollars each year, so they should be responsive to the business needs.

 

What the business expects
Then reality sets in. They want everything but they actually expect a little less than that. This varies from organisation to organisation. Some expect the top level and are somewhat delusional. Some expect the bottom level as they have zero faith in the IT department to deliver anything at all and expect all requests to go into a black hole, never to be seen again.

 

Both of these are at the tips of the bell curve though and your average organisation expects you to be responsive to their needs but realises you have limited resources (like them in most cases) and have to perform some level of prioritisation. Sure, they will whine about it but at the bottom of their hearts they know its a fact of life as often their requests directly conflict with another part of the business who may want something completely different.

 

What we actually deliver
However, what we actually delivery is somewhat less than they expect us to and is often the source or interdepartmental angst. They expect a level of prioritisation, but our opinions of where that line in the sand sits, differ. The business will prioritise in accordance to importance to them as they see it. We however, include several other pieces of work into that mix that are required to keep the lights on, that they have no visibility of, and as a result when they expect their top 5 out of 10 to be delivered, we may only deliver on their top 3.

 

What we can afford to deliver
Realistically we deliver more than we can afford and make cuts elsewhere to deliver more for the business in the fight for relevance, especially during hard times when the business is looking for budgets to cut. It’s these “cuts elsewhere” that causes us some angst later on as they may be maintenance tasks, software testing, process definition, etc. We weigh up the pro’s and con’s of doing enhancements and keeping the business happy against the risk inherent in not doing basic maintenance tasks or reducing the testing regime.

 

What we want to deliver
It really is my job just to keep the lights on for now and ever more. OS, platform and technology upgrades are “business as usual” as they make the system future proof and ensure it will work now and into the not to distant future. System enhancements are not my business. If its quick and I can squeeze it into a maintenance release then I don’t mind doing it. If its new functionality or anything that requires more than a quick regression test, then it really is out of my scope and should be the domain of the planning or projects area.

 

As a support manager, my role is to juggle and negotiate my way through this minefield of priorities and escalations and deliver something to the business that will perhaps not keep them happy but keep them content. There is no golden rule or secret formula that determines which enhancements you do and unfortunately it comes down to each particular organisation or even down to particular individuals or personalities at varying levels throughout that organisation. Often a business analyst can look deep into these requests if time permits and really determine the business value and then determine the priority based on some predetermined matrix which of course includes budget constraints along with a few other criteria.

 

So my advice in this problem is to determine how you should prioritise your enhancements, then make up a selection criteria on your own terms, cross reference them to the rest of the organisations expectations and then evaluate this against the costs involved. Its no use doing one big change that uses your entire budget surplus when you could do multiple smaller enhancements that will keep multiple units off your back for a period.

 

Good luck. You will need it.

Enhancements Requests within Application Support

Most of the discussion in the previous article was about operational issues. Something is broken, something is affected, something has gone wrong somewhere and it needs fixing now or soon.

This is known as incident management.

Apart from the usual “fixing what’s broken”, when we get time we look at old issues that results in ongoing incidents. After a while we can see patterns in incidents as they are raised and we group them into problems. We create a problem record, we perform root cause analysis and we either fix it or we don’t. If we don’t we record a known error and move on with our lives.

Interested in more? Download an ITIL book from somewhere as its all in there. We follow that framework to about half of the extent we tell people we do.

We get a multitude of requests from our end users be they internal or external to our organisation. Sometimes we get requests for items that are nothing to do with us but we helped them previously so they expect we can do everything (a common side effect of being customer focused and good at what you do).

It’s our job to keep the lights on. Sometimes that is fixing what is broken. Other times it’s replacing something that someone else has broken. When we have numerous interfaces to other organizations we can’t go through life expecting everything to stay constant. An old Greek philosopher once said “The only thing that is constant is change”. So we are somewhat at the mercy of other organisations with our budget and workloads as they do their own upgrades and decommissions and we have to adjust our systems and business processes accordingly. Sometimes with little notice. Sometimes with no budget (we call that creative accounting).

Enhancements
One of the most time consuming task that I involve myself  in is the enhancement request. End users have a myriad of ideas on a better way to do things. Some of them are valid, some of them are invalid and some of them are just “out there”. But each one of them requires (deserves) a response. The end user has taken the time out of their day to put together a request so they at least deserve a response and a purposeful one at that. As an IT department that is trying to justify its existence to the business we have to let the users know we are there and listening to what they say and more importantly responsive to what they want.

Do I do all of the enhancements? Do I do none of them? The answer is actually somewhere in the middle (but closer to the none end). My support officers record each request into our register and give it the status “Pending Review”. Then when time permits we evaluate it in more depth by asking the requester some more questions (first point of contact that shows we are listening). We may engage someone else from the business who is familiar with this area to get a “second opinion”. They may agree with the suggestion or they may not. On occasion this has revealed that its not required and is perhaps a training issue and the correct functionality already exists. Its rare that we don’t pick this up and somewhat embarrassing when we don’t since we are the super-users (but it happens, and we smile apologetically).

If it turns out to be a valid suggestion,we hand it to our development team to do a technical evaluation. At this stage it’s more of a Order Of Magnitude (OOM) assessment determining if it is a small, medium or large piece of work. This translates to free (part of a support release), within our budget capability (meaning a paid package will be generated) or expensive (meaning funding will need to be sourced and a project will need to be stood up).

Sometimes when the upper management decide we need to be more responsive they will give us some funding to do a group of enhancements on a single system. After all its cheaper to run a single block of test cases, requirements documents and implementation plans than to create one for each enhancement. So we pick a system, contact the business owners and start the process to commence a project.

I have to be clear that this then falls outside of my unit and becomes part of the project world. We aren’t part of the project world. We are part of the operations world. Generally the two only intersect during transition from project to support.

So back on topic. The developer has done their technical assessment and comes back with an assessment that says we can squeeze it into a support release. These are usually changes to a drop down menu, a screen label or some very minor functional change. It’s from here we assess the total business impact. A requester may ask for a field to be made mandatory to assist them with their reporting requirements. Sounds fairly innocuous but imagine if everyone wanted a field made mandatory. It then turns out that filling in a form that used to take 5 minutes now takes 3 hours as you have to fill in everything even if you don’t have an answer for it. Our end users will make something up and can be quite imaginative some times. Provides hours of laughs but gives the reporting guys a fit as it throws out all of their forecasts.

Once we determine its a fairly small effort and consequence we scheduled it in and get moving on it. Even if its not a small consequence we may still make a start on it. It just usually requires more stakeholder management and communication to make it a successful enhancement.

Our list of enhancement requests grow and we are currently going through some business change to only time will tell how we fair getting them all done.

In the meantime we can continue our assessments and schedule them accordingly and wait.

What does App Support do?

So what do I do in application support? Support the applications is the obvious answer. A more realistic answer is support the end users by maintaining and enhancing the tools they use to do their job.

The environment I support is a wide array of integrated applications that are built on numerous platforms, in multiple languages and not only talk to each other but across multiple other organisations to their applications.

What could possibly go wrong?

Scenario 1: Sometimes we send data to another organisation, they store it, manipulate it and when it comes back its not always in the pristine validated condition it was as when we sent it. Our system doesn’t like it any more as it no longer matches the record we have at our end and throws a hissy fit on the way back in. We have to repair it. Those are data fixes. They are frequent and numerous and make up a large portion of what our developers do on a day-to-day basis.

Scenario 2: As good as you make an application, and for all of the testing you do, there is always some scenario that you miss or don’t test thoroughly enough. The end result? Bugs. Bugs aren’t always simple to fix and aren’t always within our domain to fix. The problem could be across an inter-agency interface or embedded in a Commercial Off The Shelf (COTS) product and as a result unfixable. Or some could be just a result of the risk based testing you did (or didn’t do) and will require analysis, coding, testing and pushing through the various environments before releasing back into the wild.

Scenario 3: Patching or upgrading of underlying operating systems often introduce changes of functionality (or the application just stops working completely). Without the patches and upgrades though the operating system or COTS product may not be supported by the vendor. Any issues and the first question they ask is “Have you installed the latest updates?”. If the answer is No then the response is generally consistent across vendors. “Then that is what you need to do first”.

Scenario 4: User errors that can’t be reversed through the user interface (with generic permissions). If the user can’t fix it then it goes to a tier 1 support group who have power privileges to the application and can do things the end users can’t. Sometimes its a data fix, otherwise it may be a process that is locked up for whatever reason and the support crew (who are superusers) can remedy through their admin screen or admin account and wind back the process or just reset a status.

Scenario 5: Sometimes applications just stop responding, Sometimes they just won’t start. Sometimes they become slow beyond being usable.

Everything needs analysis before further action. And that is part of what my team does.

Apart from bug and data fixes………

Next article will be What else do we do?

Why WetSphere?

Screen Shot 2016-01-06 at 10.52.53 pm

What is a wet sphere?

It is a hard sponge type material used by florists to provide a stable foundation and some temporary sustenance.

This blog is going to “try” and perform the same function for those who work in the application support area including me. It’s my way of formulating some idea’s, explaining what I do, how I do it and why I do it and perhaps get some feedback from others in similar roles that want to bounce some ideas around or just tell me what I am doing wrong (and perhaps sometimes right) and build a solid foundation for other application support areas to work with.

As an aside I find it stimulating and somewhat relaxing to read blogs or other articles and perhaps learn something. That is my temporary sustenance as it wears off pretty quick and reality sets back in when the phone rings and there is another issue to deal with.

I will start off with What does App Support do.