When Agile Met Data Center Migration
When you hear the words “datacentre migration” you’d be forgiven if you thought agile was not the best project management fit. A methodology developed initially for software development does seem to be in a completely different universe to a project which is focussed on replacing servers and moving workloads from A to B. However, one thing you learn in consulting is to look beyond assumptions and rules and focus on finding the common patterns that run through successful projects. Agile projects are not twice as successful as waterfall projects just because teams run stand-ups every day or because someone went on a scrum course. The patterns of success we see in agile projects are driven from a deeper set of principles which are eminently re-usable across any project. Whether you are migrating datacentres or building software, once you understand those principles, you can optimise any project you work on and increase your chance of success.
But as with any set of patterns, human nature will always find a way to create an anti-pattern. One of the biggest issues we’ve seen with large scale agile transformations is the common misunderstanding that all one needs to do is follow a set of ceremonies and use a specific set of tools and an organisation will, automagically, become more agile.
We have coined this anti-pattern – “Agile Washing”.
Origins of “Agile Washing”
Here’s how this anti-pattern often starts. Someone in the C suite reads an article about how much time and money “agile” could save them without necessarily reading the fine print i.e. it is a major culture change and requires organisation-level transformation that takes time and investment and requires a fundamentally different way of measuring success. To put it bluntly, if your only success metric for agile transformation is reduced cost, you’re probably going to be disappointed for a good long while.
The C suite then kick off their annual strategy cascade and “agile transformation” is front and centre. They’re not any old C suite though, they are “informed”. They’ve read all about Objectives and Key Results (OKRs) so they know that what they need is a set of metrics/results to track this objective properly. They decide on an OKR which will be something like “x% of projects need to be agile within a year”. Satisfied they have done their job, they sit back and let the cascade do its magic.
And then the fun starts. Each head of department starts googling “agile” and tries to make it fit the world they live in. Most people end up reading about something called Scrum and after a periphery scan of the internet, they think they’ve cracked it. All they need to do is tweak their current delivery model a little, split what they do up into 2-week blocks, start using that strange Scrum board they’ve been avoiding in JIRA for the last year and send everyone on a scrum course.
The first wave of sceptical people attend the scrum course, most returning just as unconvinced as when they started, but there are some who return with notions of strange, new-fangled ideas like team autonomy and continuous delivery. Clearly, these idealists do not understand the constraints of the real world – as if things like this could ever work in their organisation! But have no fear, as soon as the legacy/waterfall PMs return from their mandated scrum master training, things return back to normal. These newly certified scrum masters know exactly how to counteract that kind of mullarky and nonsense.
Agile Washing in Action
The PMs create the backlog, estimate the tasks and assign them to the team members. It’s just like the old days but with what seems to be a lot more hours spent in pointless ceremonies like stand-ups, planning and retrospectives. But hey, if the scrum book says it must be so, it must be so.
2-week sprints are easily managed as well. High level and low-level designs are just chunked up into 2 weeks of work, easy. Same work, same order just with little breaks at the end of each 2 weeks for a few high fives and pizzas.
Tracking the project seems overly clunky though, especially with these new Jira scrum boards. When the C suite ask the PMs for guarantees on product release dates for the next 3 years, it seems too hard to do this in the scrum board, but no fear, the PMs still have their trusty MS project plan and so spend most of their time fiddling with both, re-planning mid sprint so they can make sure they stick to the perfect plan. They push the team to work overtime if they don’t finish all their tasks on time and subtly berate them for failing to stick to the story estimates that they were given. It’s heavy lifting for these Scrum Master PMs in this brave new agile world but at least their 3 year detailed plans look convincing.
And they are not alone in this confidence. The C suite tour their organisation regularly and believe they see evidence of agile everywhere. With sprint planning, stand-ups and burn-down charts wherever they look, they are mightily chuffed at their successful transformation, wondering what all the fuss was about. This was easy. Any company that fails at this is clearly not as informed as them.
However, there is a problem. Underneath this veneer of agility are the same old patterns of behaviour that caused the issues in the first place, but now things are even worse. The extra effort of agile washing compounds the problem. Costs are higher, moral is lower, deadlines are missed and quality is as bad as ever.
But at least on that next Gartner survey the C suite can tick the box that says they do agile, collect on their strategy-aligned bonus and work out what the next transformation is they should focus on. They clearly are very good at this.
That, dear reader, is classic “agile washing” and the results are not a wash you can be proud of.
Transformation with a Cultural Shift
Although the above story is slightly tongue in cheek, it is, sadly, based on an amalgamation of real experiences our team has faced when we are first engaged to help our customers accelerate their agile adoption.
In response to this, we have focussed our efforts on developing a transformation process that isn’t just lip service to agile. Our transformation process is based on the common success patterns we’ve seen when customers are truly open to this change and is aimed at triggering a ground up/top down cultural shift that will change an organisation forever.
But fair warning, this approach is not for the faint hearted. There is no going back on this transformational journey and nowhere to hide if you’re not “all in” from the start. What follows is a story of how we took agile to a place that no-one thought we could…. to the land of storage, servers and workloads, where dinosaurs still roam, and those dinosaurs eat scrum masters for breakfast.
Troubles with Data Migration
In 2017, a well-known UK bank engaged Dell Technologies to execute one of its largest programmes of work – a data centre migration of approximately 9,000 hosts and supporting infrastructure. This was a multi-year project with multiple streams and a dependency map that made every migration bundle seem like a toxic minefield. The workload migration team at Dell ran the programme as it always did, using its tried and tested waterfall delivery model. Unfortunately, despite the team having many years of experience in this field, the project was surprisingly, challenged and the causes were unclear. What was clear was that migration objectives were being regularly missed and tensions were running high.
It was at this point that someone mentioned agile. The client was already undergoing an agile transformation across some of its other departments but these infra migration projects had always seemed a no-go area for this type of approach. But something clearly had to change and agile was deemed to be the panacea for all project ailments and so it was decided that this was what was needed here.
At this point, things could have got a lot worse. An agile “wash” looked imminent as people struggled to see how this project could be transformed to agile and were confused as to what agile really meant in the context of infrastructure and migration.
It was at this critical point that our Dell EMEA Digital consulting team became involved. With years of experience of both agile anti-patterns and patterns of success, we were asked to help the program transform and improve where we could.
Successful Transition to Agile
Over a 6-month period, the programme delivery was transitioned from a traditional waterfall approach to a lean delivery model. This resulted in the team successfully meeting migration objectives for the first time in over two years. This, from a client and programme team that was understandably resistant to any change.
This was done through the pragmatic application of some core agile/lean success patterns:
- Devolution of power to programme resources.
- Reduction of waste in the value stream.
- Increased visibility of programme activities.
The Dell EMC approach was so well received by the client that it is now being replicated across the organization.
But this was more than the usual agile mantra of autonomy, visibility and lean thinking. The specific secret sauce here was much more fundamental.
We Achieved This Success by Doing 3 Simple Things:
- We identified the outcomes we want to achieve in the project
- We measured the AS IS process and identified bottlenecks/improvement opportunities
- We decided on the best improvement action for these bottlenecks and then DID IT!
Step 1 – Identifying the Outcomes
Before we changed anything, we asked the customer the simple question of “Why”? We used an outcome-oriented approach to focus the customer’s thinking on the OUTCOMES they are looking for, not just the OUTPUTS.
Outputs are just bi-products of the journey. If we focus too much on these bi-products, we may lose our direction towards the goal. Focusing on outcomes helps us measure our progress better and adapt our methods to meet our goals.
We then helped to identify the obstacles that were blocking progress and in true Stoic fashion, we reframed those obstacles into actions.
We did this in one FOTO (From Obstacles To Outcomes) workshop with the key stakeholders and from that, we jointly created a transformation plan to achieve our outcomes. The key themes of this transformation plan were as follows:
- More Transparency
- Embed an agile mindset within the team so that they can be adaptive to change.
- Create multi-skilled agile teams that can dynamically share work and take tasks from a Kanban board vs. waiting for work assignments
- Be more collaborative across the entire team
- Meet the goals of the bank on time or ahead of time
Once we all had an agreed view of where we were going, why we were going there and what obstacles we needed to overcome, we focused on a more granular discovery strategy, using what we consider to be one of the most useful tools in our Lean toolbox …
Step 2 – Value Stream Mapping
This technique allowed us to see how things work “today”, before we made any changes to the process. It allowed us to see the whole system, establish a baseline set of metrics which could then be used to measure how ongoing improvements impacted the entire delivery chain. Our simple rule of thumb was that we only change what we can measure.
The good news is that we did not need to value stream everything and we did not need to go into the most detailed levels of measurement. We picked the most common work item, or the work item that we knew was causing the most pain (in this case, a migration package) and we tracked it through each step of the current process, highlighting known bottlenecks and making less obvious bottlenecks more visible.
From this value stream map, the low hanging fruit for improvement almost always jumps off the page (or whiteboard in this case). We did not have to go into too much detail here because we were not fine-tuning a manufacturing process (which is what this technique was originally designed for). Instead, we were looking for the glaringly obvious fires, the things that everyone knew was there, but no one had acknowledged or quantified. This is a very common pattern we see when we run this value stream mapping approach. Improvement areas are easy to find once you start to look at things holistically. It’s often a human step, a manual process, a step that has ALWAYS been done but perhaps never been questioned. Changing just that one step can often have huge pay-offs to the overall speed and friction of the value stream flow. For example, involving production change approvers in verifying key technical data earlier in the process of a data center migration speeds up change approval elapsed time from weeks to days.
Once we identified the backlog of improvement items for the migration package value stream, we then started to categorise these improvement areas into the following groups – known knowns, known unknowns, unknown unknowns and chaos.
Step 3 – Categorizing the Domain and Plotting a Course for Improvement
The value stream showed lots of different types of work in the chain which could be improved but they were all at different levels of maturity. In order to improve the value chain, everyone needed to understand the maturity level of each area to help decide how best to go about that change. Not surprisingly, most of the areas of improvement fell into the “Complex” camp, otherwise known as the “unknown unknowns”, where cause and effect can only be deduced in retrospect and there are no right answers. The only sensible action for changing an “unknown unknown” is “Probe – Sense – Respond”.
For those of us that are familiar with the Build-Measure-learn approach to experimentation and innovation, this is easy to adopt. For command and control structures, this is much harder. This step itself requires a major culture shift. The acknowledgement that we have no guaranteed outcome for an action, that failure is almost an inevitable step to learning and that we are simply exploring ideas for improvement through experimentation can be a leap of faith that many find challenging.
Our advice on that is simple – start small and do not give up at the first failure. Keep cycling through the “ Baseline – Make a change – Measure the impact – Learn and Adapt “ loop and let the metrics show you the way.
The team’s faith in this empirical process eventually paid off. After 6 months of following our simple but powerful improvement plan, the team saw tangible improvements across the board.
Below are the indicators of agility of the client and delivery team. This demonstrates the collective shift in the mindset of the programme.
Every customer and project is unique and so there can be no ”one size fits all” approach to agile transformation.
As such, our EMEA Dell Digital Consulting team does not dictate a framework or measure success by counting vanity metrics such as the number of projects conducting agile ceremonies or using agile tools.
Instead, we focus on embedding core principles which allow people to adapt and improve safely but stay within the guardrails of some fundamental success patterns and we measure success using the outcomes we all agreed are important for our goals.
We create a domain relevant “agile manifesto 2.0” in people’s minds, we just don’t call it that.