Jekyll2022-04-24T16:07:48+00:00https://poros.github.io/feed.xmlAntonio Uccio VerardiPersonal github.io pageThe migration checklist2020-10-18T20:09:00+00:002020-10-18T20:09:00+00:00https://poros.github.io/migration-checklist<p><em>This is the ninth and last installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="the-migration-checklist">The migration checklist</h2>
<p>If you’ve endured this blog post series until the end, you may well be interested in a checklist to follow while running migrations, so you won’t have to read the entire thing again when you need it ;)</p>
<p><img src="/assets/images/migrations_under_monopoly.png" alt="Migration under monopoly" /></p>
<figcaption class="caption">Migration under monopoly</figcaption>
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Before you start
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Make sure you have a monopoly over your technical area</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Check whether your team can migrate everything by itself in a reasonable time</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> If you can, migrate everything by yourselves and <strong>stop</strong> here</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Preparations
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Make sure your company really wants to do this migration</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Find a use case internal to your team or create one (using the old system)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Find at least one sponsor teams with an easy use case</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Find at least one other sponsor team with a hard use case</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Alpha – You
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Write design document</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Build prototype</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Write migration plan</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Set an estimated end date for the migration (you’ll miss it)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Dark-launch your internal use case and compare with production</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Make sure your company really wants to do this migration (again)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Migrate your internal use case</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Announce the alpha release</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Beta – Sponsors
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Dark-launch the sponsors’ use cases and compare the results with production</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Write the first draft of the migration guide</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Make sure your company really wants to do this migration (yes, again)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Set up a channel to broadcast regular migration progress updates (don’t stop until the end)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Subscribe management to the above channel</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Migrate the easy use cases using the migration guide</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Migrate the hard use cases using the migration guide</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Send a migration progress update and announce the beta release</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Polish the migration guide</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Wait and help whoever asks to be migrated (if anyone)</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Automation – Bystanders
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Polish your technology until you feel confident</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Write automation to migrate most use cases without users’ intervention</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Automatically track your migration progress</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Wait</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Make sure your company really wants to do this migration (it is not too late)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> If you decided to stop, do a retrospective</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Draft a release announcement focused on the benefits for the users</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Set an estimated end date for the migration (you’ll miss it again)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Announce widely the stable release, the deprecation of the old system and your migration plan</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Stick deprecation warnings all over the old system documents and user-facing tools</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Automatically migrate all the use cases you can</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Automatically create tickets for all remaining use cases</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Send migration progress update</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Wait</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Nudges – Adopters
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Nudge all remaining users to migrate (be creative!)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Wait</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Check migration progress</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Send migration progress update</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Repeat until the number of use cases migrated per round is low and constant (or zero) for at least two iterations</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Fat tail – Laggards
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Identify the remaining use cases that must be migrated</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Remind management and the team that you have to finish this</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Tell the owners you will take care of migrating their use cases for them</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Set an estimated end date for the migration (this time, stick with it)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Communicate as widely as possible that the old system will be terminated on that date</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Migrate the must-be-migrated use cases</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Send migration progress update</li>
</ul>
</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Deprecation – Victims
<ul class="task-list">
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Shut down the old system (but be ready to roll back)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Watch the world burn</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> If it burns too hot, roll back</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Set an estimated end date for the migration (don’t give up!)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Migrate the newly discovered must-be-migrated use cases</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Send migration progress update</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Repeat until the old system has been shut down</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Delete anything even remotely related to the old system (or it will haunt you)</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Send final migration progress update</li>
<li class="task-list-item"><input type="checkbox" class="task-list-item-checkbox" disabled="disabled" /> Celebrate with your team (you have earned it)</li>
</ul>
</li>
</ul>This is the ninth and last installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: Deprecation2020-10-18T20:08:00+00:002020-10-18T20:08:00+00:00https://poros.github.io/mum-deprecation<p><em>This is the eighth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-deprecation">Migrations under monopoly: Deprecation</h2>
<h3 id="chapter-6--the-fault-in-our-stars">Chapter 6 – The fault in our stars</h3>
<p><em>The air above the Logging Infra pod was a strange mixture of excitement, anxiety and anticipation on May 4th. Luke asked for the privilege of deploying the branch to turn off Antares, and the team was happy to grant his wish. In retrospect, suspiciously happy.</em></p>
<p><em>Not that they hadn’t done their due diligence: Luke had sent out the announcement plenty of time ahead, the five must-be-migrated use cases were already running on Betelgeuse, and two more teams had migrated by themselves (not without grumbling) after hearing about the shutdown date. In case of unforeseen disaster, they had the rollback procedure ready too.</em></p>
<p><em>There was no point in feeling so anxious, really. Everything went on as if nothing had happened. For around 5 hours, 34 minutes, 22 seconds and a bunch of milliseconds. Then literally every single star known to man went dark, and nothing was left but infinite black space. The virtual space of Starwalker’s virtual space travel experiences, of course.</em></p>
<p><em>The rollback procedure brought the light back to the universe pretty quickly, but the investigation for the postmortem which followed took most of the team’s evening. The killer of stars and devourer of worlds turned out to be the legacy navigation system which powered Starwalker’s space travel. The impossibility of reading log lines with the coordinates of the space travelers convinced the navigation system that everyone had reached the very far end of the virtual universe, after which there was indeed only black. Code entropy and intricate design, all combined with the lack of a formal owner, had prevented Luke from identifying the navigation system as one of those must-be-migrated use cases.</em></p>
<p><em>Three weeks later, and with the kind help of Han and Rey, Luke was ready to try again. This time, the universe stayed bright.</em></p>
<p><em>Not everyone shared Luke and the team’s happiness and relief, though. Two teams had somehow missed this entire Antares to Betelgeuse migration thing for 16 months straight, and their workflows were now broken. Luke could not be persuaded to bring Antares back to life once again, but offered them help with migrating. They were so satisfied with the offer that Luke would never hear back from them again.</em></p>
<p><em>The day the last reference to Antares had been removed from code bases and documentation, Luke sent a card, cookies and Betelgeuse stickers to everyone who had been involved in the migration. That evening the team (yep, Han included) celebrated in the office with drinks and funny stories. At the end of the night, Luke was genuinely surprised when Leia passed him one of those DeathStar VR headsets. He couldn’t help but smile as Antares lit up in a shiny supernova.</em></p>
<p><img src="/assets/images/migrations_under_monopoly_6.png" alt="Deprecation" /></p>
<figcaption class="caption">Deprecation – Victims</figcaption>
<h3 id="deprecation--victims">Deprecation – Victims</h3>
<p>The last and final stage is the one where you are going to pull the weight of your <strong>monopoly</strong> to reach the <strong>termination</strong> (a.k.a. it won’t run anymore), or at least the complete <strong>deprecation</strong> of the old technology, meaning that no more maintenance time will be spent on it and that it could stop working at any time.</p>
<p>I call the users of this final stage <strong>victims</strong>, as their use case will be sacrificed for the good of the migration and the company. These may be laggards whose use case was not important enough for your team to migrate in the previous stage, or people who are downright hostile to the new technology and who would never have migrated. They may end up switching to a competitor snowflake system or deprecating their use case altogether, but the cost will be localized to their team and not spread across the entire company for years to come, making it much easier to manage in the future.</p>
<p>On the date you set in your announcement, <strong>shut down the old system</strong> (or stop maintaining it), but be ready to <strong>roll back</strong>. As well as being good practice, this is necessary for the migration, so make sure you know how to do it.</p>
<p>Now <strong>watch the world burn</strong>. If nothing burns too hot (or at all), congratulations, you did a great job! If something burns too hot, <strong>roll back and don’t give up</strong>!</p>
<p>You should have now <strong>discovered</strong> one or more must-be-migrated use cases that you missed the first time. This is incredibly common in large companies, where it’s practically impossible to gather all the context about a certain use case and understand how many things actually depend on it.</p>
<p>Set a <strong>new</strong> deprecation date, migrate the newly discovered must-be migrated use cases, send a migration progress update and try again. <strong>Repeat until the old system has finally been terminated or deprecated</strong>.</p>
<p>Don’t celebrate just yet. You have to **clean up **everything even remotely related to the old system: code, monitoring and especially tutorials or documents that could make someone resurrect the old system by mistake. Don’t skip this step, or the old system will come back and haunt you!</p>
<p>Now you can send the <strong>final</strong> migration progress update and <strong>celebrate with your team</strong>: you have earned it! :)</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/migration-checklist/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">The migration checklist</figcaption>
</a>
</div>This is the eighth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: The fat tail2020-10-18T20:07:00+00:002020-10-18T20:07:00+00:00https://poros.github.io/mum-the-fat-tail<p><em>This is the seventh installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-the-fat-tail">Migrations under monopoly: The fat tail</h2>
<h3 id="chapter-5--children-of-a-dying-sun">Chapter 5 – Children of a dying sun</h3>
<p><em>“Luke, you did plenty already, are you really sure we have to go ahead with this migration?” Leia remembered as soon as she saw the smile widening on Luke’s face.</em></p>
<p><em>“You know that we have to finish it. We are paying to run both Antares and Betelgeuse now, the time going into maintenance has doubled, people have to learn about both frameworks, and Antares will be a drag for any future innovation,” Luke reminded her.</em></p>
<p><em>“I’ll be honest, GalacticEmpire Inc grew up to be a pretty worrying competitor in the space of… well, virtual space travel. Their latest feature, DeathStar, gives customers the opportunity to see how their favorite star would evolve at super-high speed, and become a supernova or a red giant.” Leia wanted Luke to see where she was coming from. “The board considers Project Force our answer, and we need all the help we can get to release it next quarter. If Rey plus someone else on the team could give a hand, even temporarily…”</em></p>
<p><em>Luke let her finish. “May 4th.” He did pause for dramatic effect. “May 4th is the day we shut down Antares. I have already told Rey that she is free to go help with Project Force. She said she wants to see this to the end, but I am sure you’ll persuade her. You have your way with people.”</em></p>
<p><em>Luke had identified 5 use cases out of the 17 left in the fat tail which must have migrated to Betelgeuse before they could turn off Antares. Missing even one of them would have resulted in critical failure for many of the core systems of Starwalker. The team was not exactly thrilled to work on five old pieces of software it did not own, but did prefer it to the prospect of maintaining both Antares and Betelgeuse forever. The owner teams, instead, were perfectly thrilled about not having to do the migration themselves. If you can afford it, waiting things out is a pretty effective negotiation strategy, Luke had to admit.</em></p>
<p><img src="/assets/images/migrations_under_monopoly_5.png" alt="The fat tail" /></p>
<figcaption class="caption">The fat tail – Laggards</figcaption>
<h3 id="fat-tail--laggards">Fat tail – Laggards</h3>
<p>The following stage is the one where most abandoned migrations end up beached, because of the lost momentum and because the company’s interest has moved on to something new and shinier which shows a lot of promise. I refer to this stage as the <strong>fat tail</strong> of use cases which, for one reason or another, have never been migrated and will likely never be. The users’ profile for this phase is that of the <strong>laggards</strong>: they are not against the new technology, but they really don’t find it compelling enough to prioritize adoption over everything else, and they ended up lagging behind. They might eventually migrate, though; in fact, in this phase you may still see some use cases migrated over time, but not nearly enough to finish the migration in a reasonable time.</p>
<p>The way to navigate out of this phase of excessive calm and probably low morale of your team is <strong>to act</strong>.</p>
<p>Start by identifying all the remaining use cases that <strong>absolutely must be migrated</strong>. By this I mean revenue-critical applications or widely used products – things that your company cannot afford, or doesn’t want, to live without. Bring your freshly assembled list to management and your team, reminding them that you <strong>must finish this</strong>, to avoid incurring all the costs and pains of a failed migration that we’ve already discussed. Finally, reach out to each of the owners of these must-be-migrated use cases to tell them that your team will take care of migrating their use cases for them. I know this sounds frightening, but you must finish this, remember?</p>
<p>Set an estimated end date for the migration (you probably missed your previous one), and this time <strong>try to stick to it</strong>. The reason is that you will communicate as widely as possible that <strong>the old system will be terminated on that date</strong>. And this is a promise you’ll want to keep if you want to maintain enough credibility to do what comes next.</p>
<p>Now migrate one must-be-migrated use case after the other, and send migration progress updates regularly to build up momentum and expectations for the end of the old technology and the migration altogether.</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-deprecation/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: Deprecation</figcaption>
</a>
</div>This is the seventh installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: Nudges2020-10-18T20:06:00+00:002020-10-18T20:06:00+00:00https://poros.github.io/mum-nudges<p><em>This is the sixth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-nudges">Migrations under monopoly: Nudges</h2>
<h3 id="chapter-4--lights-from-the-past">Chapter 4 – Lights from the past</h3>
<p><em>Luke was coming out of each team meeting a little more tired than the one before. He always found it very hard to keep people motivated during periods of prolonged calm, perhaps because he himself was losing motivation. The migration was dragging out, and all the momentum they had built up to the peak at the end of last year had been slowly eroded by six months of attrition.</em></p>
<p><em>And the waiting. The waiting was the worst part for a man of action like him. He had promised to himself that he would keep up sending progress update emails, together with a nudge every two weeks, but he hardly had anything to report this time. The “Betelgeuse party” they threw right after the bulk migration had been both fun and a big success, with eight use cases migrated in the same month. The “Betelgeuse challenge”, with a prize of T-shirts featuring the Betelgeuse logo, got them five more adopters. After that, his creativity kinda lost its edge. Targeted presentations didn’t work out very well. His last two nudges had been stuff like “You can see that more than 95% of the company has already forgotten Antares’ problems” or “All the other teams in the Infra group are already on Betelgeuse”, to leverage the sense of guilt of the remaining teams. Based on the results, they must have been gangs of unredeemable criminals.</em></p>
<p><em>He was still struggling to come up with something for the next nudge when he noticed Leia and Rey chatting over tea in the kitchen area. “I would be so down to help out with that project, Leia!” – open spaces must have legally de-penalized eavesdropping, Luke thought – “But we maintain both Antares and Betelgeuse now, and that’s a ton of work. At the pace of one use case a month, we’ll never truly get rid of Antares.”</em></p>
<p><em>Luke opened the migration tracker once again, and this time he was able to see it clearly: they had reached the fat tail. It was again time for him to act. It’s funny that all he needed to realize it was just a little nudge.</em></p>
<p><img src="/assets/images/migrations_under_monopoly_4.png" alt="Nudges" /></p>
<figcaption class="caption">Nudges – Adopters</figcaption>
<h3 id="nudges--adopters">Nudges – Adopters</h3>
<p>The next phase of a migration under monopoly is the one of <strong>nudges</strong>, which we are going to use in order to compensate for the loss of momentum typical of the second half of migrations. I refer to the users targeted in this stage as <strong>adopters</strong>, as they are going to make a concrete effort to adopt your new technology for their use cases. It’s easy to feel frustrated because of not-very-cooperative users or teams constantly delaying previous commitments. But appreciate that these people are still taking action to migrate to a new system which has been imposed on them (you know, monopolies), and it is very likely that their use cases are non-trivial as well, as you couldn’t migrate them automatically in the previous phase, so feel grateful that they have not decided to ignore your requests instead.</p>
<p>The way to lead this stage of the migration forward is to establish a cycle of nudges, checks and updates.</p>
<p>As the first step of the cycle, send a <strong>nudge</strong> to the users who still have to migrate, to remind them that they should find some time to get through your migration guide and that you are happy to help. Nudging people effectively is an art, and being <strong>creative</strong> with your nudges can get you a long way. You can use wide-reaching channels such as speaking at your company live events, or you can reach out to each individual user privately (you should have a ticket for each of them already, so it should be fairly easy to keep track of what you’ve done). You can try to explain to them how this new feature of the system would make their life better; or you can use more subtle techniques, like telling them that everyone else in their group has migrated already. You can offer to embed an engineer in their team for the duration of the migration, or give them a voucher for a migration they will be leading soon. Be creative!</p>
<p>After the nudge comes the <strong>waiting</strong>, and after that you can use the <strong>migration tracker</strong> you built previously to check the progress of the migration and to send a <strong>progress update</strong> through your communication channel. This update will also work as a nudge to other future adopters!</p>
<p>And then repeat. Repeat until you find that <strong>nudges don’t work anymore</strong>. This is easy to get wrong, and it is a pretty common mistake to either waste time in an endless loop of nudges or advance the migration to the next stage prematurely, with a much higher cost for your team than necessary. I would advise that you use your migration tracker to <strong>check</strong> that the number of migrated use cases per iteration is low and constant, or even zero, for at least two iterations before moving to the next phase; but the best exit condition really depends on your context.</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-the-fat-tail/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: The fat tail</figcaption>
</a>
</div>This is the sixth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: Automation2020-10-18T20:05:00+00:002020-10-18T20:05:00+00:00https://poros.github.io/mum-automation<p><em>This is the fifth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-automation">Migrations under monopoly: Automation</h2>
<h3 id="chapter-3--the-sun-is-closer-from-up-here">Chapter 3 – The sun is closer from up here</h3>
<p><em>“379. Is that a prime number? It’s gotta be a prime number.” Rey’s mind was toying with what was showing up in the migration tracker. “379 repositories left to migrate to Betelgeuse versus… well, 5. It’s kinda hard to test the regular expression with only 5…”. Her thought was interrupted by Luke entering her field of vision.</em></p>
<p><em>“Hey Luke, I think I am practically done with the automation for the migration. I’ve run the code refactoring tool in dry run mode on a couple large repos, and with this set of rules I managed to switch to the new reader API for…”.</em></p>
<p><em>Rey was the most promising engineer in Luke’s team. On the good days, also the most talkative: “… run a modern kernel; with that option we should be able to bind the local Betelgeuse agent to the same port of Antares transparently from the client perspective and without losing in-flight messages as planned in the design phase. Have you got any questions?”</em></p>
<p><em>“379.” Luke’s cunning smile widened as he watched Rey’s reaction. “Is that a prime number?”</em></p>
<p><em>“It’s gotta be a prime number.” Leia could not resist mocking Luke a little.</em></p>
<p><em>“It’s a big number.” Luke knew how to take a joke, but not this time. “And among them there are plenty of legacy code bases. Rey is great, and the automation she built will work for most repositories, but some of them will have to be migrated manually. From what she says, we could end up with thirty or more situations pretty much like the billing workflow one. This is definitely going to bleed into the next two quarters.” Luke was going at full steam. “I know that rolling back would be pretty rough now after all this work we put in, but it’s still an option. Are we sure that we really want to do this migration?”</em></p>
<p><em>“Yes, this is still very important for the company and I believe that we should move forward.” Leia didn’t like the fact that her voice sounded a little annoyed. “I appreciate you being considerate, but how many more times are you gonna ask me that?”</em></p>
<p><em>“That was the last one. What about you ask me next time?” Luke’s distinctive grin was back on his face.</em></p>
<p><em>“I will if you take up the whole team for yet another year.” Somehow Leia liked him more that way. “You have five minutes at the next All Hands.”</em></p>
<p><em>“With Betelgeuse, you can choose between guaranteed event delivery and low-latency logging, but in our tests even the latter has a delivery rate 100 times better than Antares. It’s been powering five production use cases for months, and we measure the uptime in the five nines.” Luke had rehearsed the sales pitch many times, but speaking to the entire company still made him a bit nervous. “We are going to migrate more than 90% of the use cases automatically in the coming weeks, so most of you won’t even have to do anything! With your help, we plan to be entirely off Antares by the last quarter of the year.”</em></p>
<p><em>Looking down at the 32 tickets tracking the repositories which required manual migration, Luke was not too sure about that “last quarter of the year” anymore. Looking down from the height of those 347 use cases migrated in less than three weeks, of course. 347. Was that a prime number?</em></p>
<p><img src="/assets/images/migrations_under_monopoly_3.png" alt="Automation" /></p>
<figcaption class="caption">Automation – Bystanders</figcaption>
<h3 id="automation--bystanders">Automation – Bystanders</h3>
<p>The third stage is the one of <strong>automation</strong>, as it is going to play a key role in migrating the <strong>bulk of users</strong> from the old technology to the new one. The users of this phase have well-understood and rather simple use cases at this point, and they are (hopefully) the vast majority. These users are going to stay mostly <strong>passive</strong> and only watch the migration unfolding under their eyes through your chosen channel of communication, and for these reasons I call them <strong>bystanders</strong>. This is also going to be the peak of your migration, so be ready for the hike.</p>
<p>To climb up the peak, it’s time for you to consider investing in <strong>automation</strong>. In my own personal statistical sample, this is the single most neglected part of migrations, probably because you might be able to get by without it. The returns for your team and company are enormous, though, so I would deem it shortsighted to cut automation down in order to save budget or time.</p>
<p>The first automation you want to invest in is <strong>automating away as much as possible of what is written in your migration guide</strong>. To begin with, a quick and dirty prototype is enough, but your ideal goal is to eventually be able to <strong>automatically migrate most use cases</strong>. For the best results, I would recommend factoring this into the <strong>initial design phase</strong>.</p>
<p>The second automation you want to invest in is an automatic <strong>tracker of your migration progress</strong>. This can be very easy, like a list and a counter of migrated use cases over all the ones running on the old system. If your company has been around for a while, this will likely already exist. If it doesn’t, many future engineers will be in your debt.</p>
<p>Before getting into the weeds of this migration phase, take some time to <strong>polish</strong> your product and your migration automation while you <strong>wait</strong> for the feedback from the beta users. Waiting might feel excruciating when you’ve gained so much momentum from the beta stage, but it is giving your users and management breathing room to establish <strong>trust</strong> in the new system just by using it and seeing it running. If you still have some lingering reliability problems, now is probably the best moment to iron them out before going full throttle. Waiting time is also a good time for retrospection.</p>
<p>I have built into my recipe for migrations a good deal of escape hatches to avoid the catastrophic failure of a never-ending or abandoned migration, and this one is the <strong>last chance</strong> you have. Be absolutely certain that your company <strong>really</strong> wants to do this migration: <strong>it isn’t too late</strong>. Failing now will have a non-negligible cost, with three production use cases to roll back, a probably compromised trust relationship with your sponsors, and a big blow to your team morale and motivation. But, trust me, this is nothing compared to what your company and your team are going to spend over time maintaining the complexity of two systems that no one is satisfied with until a third, harder, migration comes over. Don’t be scared of throwing away months of development time: those are <a href="https://en.wikipedia.org/wiki/Sunk_cost">sunk costs</a> and you’ll never get them back. You, your team and management should rely on each other to make the best decision with the data you have.</p>
<p>If you have decided to stop here, do a <a href="https://en.wikipedia.org/wiki/Retrospective">retrospective</a> to learn how to fail faster next time. If you have decided to move forward, now don’t give up until the very end.</p>
<p>It’s time to draft a release announcement for your <strong>stable</strong> release of your product, where “stable” simply means “no major bugs, reliable and performant, interfaces will change rarely and in a backward-compatible way, used in production”. Some products enter this stage in what people call <strong>perpetual beta</strong>, which is exactly like before minus the stable interfaces assurance to allow for faster development cycles.</p>
<p>What you want to go for is up to you, but I would strongly advise you to draft this announcement focusing on the <strong>benefits for the users</strong> of the new system compared to the old. You want to convince people to migrate (more about this in the next stage), and your best shot at it is now, when all eyes are going to be looking at you. Ah, and don’t forget to set a new estimated end date for the migration: you missed the old one, remember? Don’t worry, it’ll happen again.</p>
<p>Once you are ready, announce as <strong>widely</strong> as possible the stable release of the new system, your migration plan and the <strong>deprecation</strong> of the old system. It’s also a good time to stick deprecation warnings all over the old system documents and user-facing tools (doing it earlier would confuse users).</p>
<p>Now it’s time to execute your migration plan and <strong>migrate all the users with simple use cases</strong> using the <strong>automation</strong> that you built and used in the past. You can do this in batches if it makes it easier, of course. Keep sending migration progress updates, relying on the information shown by your automatic <strong>tracker</strong>.</p>
<p>Usually something weird or wrong comes up for a small percentage of use cases at this point. <strong>Create tracking tickets</strong> for them, as for all the other <strong>more complex use cases</strong> which could <strong>not</strong> be migrated automatically, and move them to the next phase. But before going there, let’s wait a bit to establish <strong>trust</strong> again: you’ve just migrated most of the users!</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-nudges/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: Nudges</figcaption>
</a>
</div>This is the fifth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: Beta2020-10-18T20:04:00+00:002020-10-18T20:04:00+00:00https://poros.github.io/mum-beta<p><em>This is the fourth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-beta">Migrations under monopoly: Beta</h2>
<h3 id="chapter-2--reach-for-the-stars">Chapter 2 – Reach for the stars</h3>
<p><em>The dark launch of the Stars Catalog service was running so smoothly that it was hard for Luke to persuade Finn to let it marinate a little more. Not that Luke wasn’t confident in the stability of Betelgeuse, but the migration guide that he was drafting needed a bit more love before any user could follow it.</em></p>
<p><em>Instead, what Luke had little confidence in was his ability to find a compromise with the Revenue team and its manager, Lando. Since the beginning of the dark launch of its legacy billing workflow, the team had kept on discovering weird ways in which its application was relying on undocumented behaviors of Antares. Today’s meeting was about one exotic partial ordering property of log lines which had simply no place in a distributed framework like Betelgeuse. Luke was ready to listen to some horrible hacks, when he walked into the room and found Han drawing on the whiteboard.</em></p>
<p><em>Han had been a remote developer at Starwalker for years, and a brilliant one, but Luke had heard only legends about his last visit to the office.</em></p>
<p><em>“Lando said that if I can find a way to make it work with Betelgeuse, I am free to re-architect the entire billing workflow the way I like,” Han explained without stopping to draw… “I hate that thing.”</em></p>
<p><em>“Well, that makes two of us!” replied Luke, picking up a marker.</em></p>
<p><em>Two months later, the new billing workload had already been dark-launched.</em></p>
<p><em>“That’s one cool story, Luke,” Leia said, looking up from her laptop. “And the metrics are looking good and the teams are happy. So why are you here?”</em></p>
<p><em>“I wanted to double-check with you that we still want to go ahead with Betelgeuse.” Luke was unusually serious. “The migration guide is ready. We could ask our sponsors to migrate today, if we wanted to.”</em></p>
<p><em>“Listen, you seem to have decided already.” This time it was Leia who could barely hide a smile. “Also, no one here would have the guts to tell Han that his new project wouldn’t be going to production.”</em></p>
<p><em>The first email that Luke wrote to the freshly minted “Betelgeuse-users@starw.com” mailing list was the announcement of the beta version of Betelgeuse, an update on the two successful production launches for the sponsor teams and a pledge for beta testers. Attached to the email there was also a three-minute video of Han in his living room saying a few good things about Betelgeuse. And about the new billing workflow, of course. Leia could not hide her smile this time.</em></p>
<p><img src="/assets/images/migrations_under_monopoly_2.png" alt="Beta" /></p>
<figcaption class="caption">Beta – Sponsors</figcaption>
<h3 id="beta--sponsors">Beta – Sponsors</h3>
<p>I call the second stage of a migration (unsurprisingly) the <strong>beta</strong> stage, taken from the name of the final deliverable of this phase. This is usually the one that exhibits the most variance according to who is leading the migration, but you should be able to recognize some key standard practices in my personal recipe if you have already been in a migration or two.</p>
<p>The users of this phase are usually referred to as “early adopters”, but I much prefer the name that Will Larson came up with: <strong>sponsors</strong>. These people’s profile is the one of engineering teams who could really benefit from, and are excited about, the new technology you are promising; and they are ready to prioritize helping you out over some of their core business. They are going to give you not only precious time and feedback, but also <strong>visibility</strong>. Their successful use cases are going to be the best advertisement you’ll ever get for your migration, both with future users and with management. Treat them well – they are making a bet on you!</p>
<p>While your team keeps up with the usual iterations of development, the first thing you want to do in the beta stage of the migration is to work with your sponsors to <strong>dark-launch</strong> their use cases. Now, if you have selected them as I advised in the Preparations phase, the first use case should be pretty straightforward to launch, while the second one should be way more complex and troublesome, hopefully pushing your team to change and evolve your product to adapt to a much broader set of use cases. Monitor the same metrics that were used for your internal dark launch, or a superset of them, to compare your sponsor’s dark-launched use cases with the ones running on the old system until everything looks right.</p>
<p>This is also the time to use all the experience accumulated so far to write the first draft of your <strong>migration guide</strong>, which will be used by your future users to migrate their own use cases. This should also give you an idea of how hard it is going to be for users to migrate, and whether there are any pieces of the migration procedure which could be automated away. More on this in the next blog post. A useful addition to the migration guide would be a section describing how to roll back the migration in case of any problem.</p>
<p>You should now have a pretty good idea of how hard this whole migration business is going to be, so it’s time to re-evaluate whether your company <strong>really</strong> wants to do this migration (yep, one more time). Your sponsors, your team and management have all the data they need to identify a very risky migration and make it <strong>fail</strong> when the rollback cost is still pretty low. There is only your internal use case to roll back, which you could do on your own, and your sponsors’ production use cases have not been touched yet. Listen carefully to anyone who has cold feet. If you fail now, you will save everyone a good deal of prolonged pain.</p>
<p>This is also a good time for you to step up your <strong>communication</strong> game. Set up a <strong>channel to broadcast regular updates</strong> about the progress of the migration (what the best channel is going to be depends on your company). It’s also a good idea to <strong>subscribe management</strong> to that channel. And <strong>don’t stop sending updates until the very end.</strong> What kills most migrations and leaves them in the limbo of never-ending endeavors is that <strong>people forget about them</strong>.</p>
<p>At this point, you should have had enough time to nail that old dark launch business, so be ready to finally migrate your sponsors’ production use cases. Actually, <strong>ask</strong> your sponsors to migrate them using your migration guide. This should be a good source of feedback.</p>
<p>Assuming everything goes well, it’s time to send the first <strong>migration progress update</strong> and <strong>announce the beta release</strong> of your system. The expectations for a beta version are likely going to be “no known critical bugs, features are missing, interfaces may change in a non-backward-compatible way, but this should pretty much work in production”.</p>
<p>In the announcement, remember to make clear that <strong>beta testers</strong> are welcome and that you are eager to help them out. And now <strong>wait</strong>. You may not get anyone who wants to migrate early, but the feedback you’ve got from your sponsors should be enough to <strong>polish</strong> your migration guide and build some automation for the next stage.</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-automation/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: Automation</figcaption>
</a>
</div>This is the fourth installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: Alpha2020-10-18T20:03:00+00:002020-10-18T20:03:00+00:00https://poros.github.io/mum-alpha<p><em>This is the third installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-alpha">Migrations under monopoly: Alpha</h2>
<h3 id="chapter-1--shiny-little-things">Chapter 1 – Shiny little things</h3>
<p><em>It didn’t take many sprints for the Logging Infra team to get a prototype of Betelgeuse up and running. Sure, it was held together by duct tape and good intentions, but the design document Luke and the team had produced was already showing the light of a much brighter future.</em></p>
<p><em>It wasn’t any of the flashy architecture diagrams which caught Leia’s eye, though. What she was discussing with Luke at that moment was the migration plan that he had shared with her earlier that afternoon. Actually it was just this one bit of the migration plan.</em></p>
<p><em>“Are you really sure that this is going to take the whole year?” she asked.</em></p>
<p><em>“That sounds a bit too close, doesn’t it?” Luke replied with half of a smile.</em></p>
<p><em>“We both know that it isn’t what I meant,” Leia said, her eyes narrowing.</em></p>
<p><em>“Eh, don’t give it too much thought. We’ll have plenty of time to tweak the estimates as we move forward.” Luke’s tone changed back to serious. “We are sure that we actually want to move forward, right?”</em></p>
<p><em>A dark-launched version of Abacus had been logging to Betelgeuse in production for about two weeks now, and all the metrics were looking great. Compared to the Antares instance, message loss was practically zero, latency down by one order of magnitude. But Leia had been a CTO for long enough to know that Luke wasn’t just worrying over nothing. Lots of things could change in one year, and a team of six engineers might come in handy one day. Still, the numbers were pretty good and everyone was so excited… “Of course we want to move forward. Ping me when the team hits prod for real.”</em></p>
<p><em>It wasn’t long before she heard back from Luke. And, together with her, the Star Catalog and Revenue teams.</em></p>
<p><em>“We are still ironing out a couple bugs but, thanks to the experience we gained from both the dark and the actual production launch of Abacus, the team has refined Betelgeuse so much already that it’s almost offensive to call this an alpha version.” Luke wondered for a second if that sounded too cheesy. Just for a second, though.</em></p>
<p><em>“Luke, so… can we start using Betelgeuse yet?” The young tech lead on the Star Catalog team couldn’t wait for the Q&A at the end.</em></p>
<p><em>“I’ll stop by your desk right after this, Finn.” Luke had to admit that Finn wasn’t the only one excited.</em></p>
<p><img src="/assets/images/migrations_under_monopoly_1.png" alt="Alpha" /></p>
<figcaption class="caption">Alpha – You</figcaption>
<h3 id="alpha--you">Alpha – You</h3>
<p>The first stage of the migration takes its name from its final deliverable: the <strong>alpha</strong> version of the new technology. You won’t necessarily build the new system yourself, but even if you buy it directly from a vendor, there will always be some integration work necessary to make it fit within your company’s technology stack and processes. That’s what your team is going to deliver.</p>
<p>As for the profile of the users of this first phase, that’s pretty easy: <strong>it’s you</strong>. In the early stages of iteration, you’ll need a user who is expert in the domain, brutally honest, heavily invested in the migration and with the closest feedback loop possible. Obviously there is no one fitting this description better than your own team.</p>
<p>In the alpha stage, you are going to follow your standard practices of good software development, but in general you should at least come up, in a couple of iterations, with some sort of <strong>design document</strong> and a <strong>prototype</strong> capable of running the internal use case you selected in the Preparations phase.</p>
<p>As part of or alongside the design document, you should also draft a <strong>migration plan</strong>, describing how you intend to port <strong>all</strong> the company’s use cases to the new technology. I know it sounds like a daunting task, but don’t worry, you can use the checklist at the end of this blog post as a starting point. And don’t forget to set an <strong>estimated date of completion</strong> for the migration. Don’t try too hard to get it right – chances are that you’ll miss it anyway.</p>
<p>Once you are all set, you should be at the stage of being able to <strong>dark-launch</strong> your use case on the new system and <strong>compare</strong> it to the one running in production on the old system. <a href="https://www.ibm.com/garage/method/practices/run/practice_dark_launch_feature_toggles/">Dark launches</a> (or soft launches, or feature toggles, or canaries – choose whatever best fits your use case) are a very valuable tool during the early stages of a migration, as they will allow you to run your new system in <strong>production</strong> for validation, segregated from your original use case and with little to no impact on customers. We’ll rely on them again in the next stage.</p>
<p>With the experience and the metrics you have collected from the dark launch, your understanding of the technological challenges should be much deeper than at the beginning, so it’s time to re-evaluate whether your company <strong>really</strong> wants to do this migration. <strong>This is the best time to fail your migration</strong>. The cost of failing fast is orders of magnitude lower than that of failing at a later stage of a migration, when giving up would mean reverting all the already migrated use cases or, even worse, maintaining two systems you are not satisfied with at the same time. And that’s without considering all the future sunken development costs.</p>
<p>Assuming you want to go ahead, it’s time to <strong>migrate your use case</strong> for real. This usually goes reasonably well, as you have designed your prototype around it. During all this time your team will have improved (or rebuilt) your prototype into a product thanks to all the insight gained, the bugs discovered and the time passed since the start of the project. Make sure you keep this up until the last couple of stages of the migration.</p>
<p>Now you are ready to announce to users and stakeholders that the <strong>alpha</strong> version of your shiny new thing is available! You may not consider this such a big deal after the software industry shifted to continuous delivery, but releases still play the crucial role of setting the <strong>expectations</strong> that your users will have for your product. Alpha should be perceived along the lines of “it’s buggy, interfaces are going to change in a non-backward-compatible way, but it kinda works”, and “kinda works” is as big as the distance between possible and impossible. Congrats! You can move to the next stage.</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-beta/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: Beta</figcaption>
</a>
</div>This is the third installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.Migrations under monopoly: Preparations2020-10-18T20:02:00+00:002020-10-18T20:02:00+00:00https://poros.github.io/mum-preparations<p><em>This is the second installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="migrations-under-monopoly-preparations">Migrations under monopoly: Preparations</h2>
<p>In this post and the ones to follow, I will go over my ideal model of large-scale migrations under monopoly and my personal recipe for how to run them successfully. To do so, we will follow the fictional migration of all software applications in the Starwalker company from their first-generation in-house logging framework, Antares, to their second-generation one, Betelgeuse. But before we start, let me tell you what this post will <strong>not</strong> cover.</p>
<h3 id="start-from-the-basics-somewhere-else">Start from the basics (somewhere else)</h3>
<p>If you are <strong>completely new</strong> to technology migrations, I honestly recommend you <strong>leave</strong> this place to read <a href="https://lethain.com/migrations/">Migrations: the sole scalable fix to tech debt</a> by <strong>Will Larson</strong> and/or watch his talk <a href="https://www.youtube.com/watch?v=OFjvJmS_uDo">Paying Technical Debt at Scale – Migrations @Stripe</a>. He is a much better writer than I am, and I learned a ton from the content he produced. Once you can articulate to your boss and your users <strong>why</strong> you’d want to run a migration, please be my guest again :)</p>
<p>I will also assume that:</p>
<ul>
<li>you are an expert in your technology domain</li>
<li>you are used to software development</li>
<li>you know how to build “the right thing”</li>
<li>you know how to keep a high level of quality while operating with a finite budget, time and staff.</li>
</ul>
<p>I know this is a lot to take for granted, but I want to keep this post focused and not turn it into a book :P</p>
<h3 id="chapter-0--a-new-star-is-born">Chapter 0 – A new star is born</h3>
<p><em>At Starwalker, the world-leading company for space travel virtual experiences, there is a running joke that if you chat for long enough with someone working in engineering, they will eventually start complaining about Antares. Developers hate it because it’s hard to use, it’s excruciatingly slow, and log lines often go missing. Maintainers hate it because it’s a hot mess of legacy infrastructure and breaks every other night. Management hates it because it costs a ton of money to keep the servers running and it’s hard to find people who want to work on it.</em></p>
<p><em>When Luke from the Logging Infra team pitched a new-generation scalable logging framework which would fix all the users’ complaints and would cost a fraction to run, the room could barely contain its excitement. Luke was going over the costs and risks involved in such a potentially huge effort, without doubt the biggest migration in the history of Starwalker to this day, but Leia, the CTO, had already made up her mind ten slides ago: Project Betelgeuse was born.</em></p>
<p><em>There were plenty of teams with modern web services who wanted to get their hands on a prototype of Betelgeuse as soon as possible, and the Star Catalog team just happened to be the largest one. But convincing the Revenue team to let Luke use its ancient payments collection pipeline as a guinea pig for Betelgeuse, now <strong>that</strong> wasn’t easy. Luke had to promise to offer strong delivery guarantees from the get-go, so that the team could keep extracting billing events from its logs. That said, the Revenue team was already losing 3% of those events every day (it even logs them twice!) and everyone knows how good an incentive 3% of revenue can be.</em></p>
<p><em>While Luke was securing high-profile sponsors for the Betelgeuse project, the Logging Infra team had just shipped Abacus, a simple application which was continuously logging a bunch of lines to just read and count them every two minutes. People looked a little surprised when they heard it was still using Antares.</em></p>
<p><img src="/assets/images/migrations_under_monopoly_0.png" alt="Preparations" /></p>
<figcaption class="caption">Preparations</figcaption>
<h3 id="preparations">Preparations</h3>
<p>Preparatory work doesn’t show up on the chart, but it is foundational to successful migrations and establishes the relationships with the three actors in migrations: <strong>the boss, the users and the team</strong>.</p>
<p>First and foremost, you must make sure that your company <strong>really</strong> wants to embark on the migration. Migrations are long commitments with pretty unpredictable end dates and are hard to justify in terms of return on investment, so make sure that management is on-board and aware of all risks and costs. You’ll need all the support you can get going forward.</p>
<p>While you talk with management, you want to find (at least) <strong>two users</strong> who are excited about the migration and would like to trial it early on (later we’ll call them <strong>sponsors</strong>). There are additional constraints: at least one of them must have a pretty <strong>straightforward use case</strong> to migrate, while at least one other must have a pretty <strong>hard and complex one</strong>. In addition, I would strongly recommend that they are very significant in terms of <strong>impact</strong> for the company (e.g. the Revenue team), to maximize your early visibility. We will go over the reasons behind these additional constraints in one of the following blog posts.</p>
<p>The selection of the sponsors is by far one of the hardest parts of a migration, as it is pretty hard to find two of them respecting all the criteria. But don’t cheat – this is important. And if you cannot manage to find them, are you really sure that this migration is truly what your company should be focused on right now? :P</p>
<p>Lastly, make sure that your team has at least one <strong>internal use case</strong> that could be migrated to the new technology; and if it doesn’t, <strong>create</strong> one running on the <strong>old</strong> technology you want to migrate away from. <a href="https://en.wikipedia.org/wiki/Eating_your_own_dog_food">Dogfooding</a> is exactly what we are going to focus on in the next stage.</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-alpha/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: Alpha</figcaption>
</a>
</div>This is the second installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.A taxonomy of migrations2020-10-18T20:01:00+00:002020-10-18T20:01:00+00:00https://poros.github.io/taxonomy-of-migrations<p><em>This is the first installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts <a href="https://poros.github.io/technology-migrations-series/">here</a>.</em></p>
<h2 id="a-taxonomy-of-migrations">A taxonomy of migrations</h2>
<p>When we talk about migrations in the technology domain, we usually refer to the process of moving from one technology, system or way of operating something to another. Within this pretty broad definition fall quite a few things that look pretty different from each other, so there is a lot of space for sorting them out into various categories.</p>
<p>My personal <a href="https://en.wikipedia.org/wiki/Taxonomy">taxonomy</a> of technology migrations is based on two discriminants: <strong>market type</strong> and <strong>scale</strong>. Rather unsurprisingly, the scale axis goes from small to large, while the market type is on the spectrum between competitive and <a href="https://en.wikipedia.org/wiki/Monopoly">monopolistic</a>. Other than the fact that I personally find it amusing, this classification helps me to communicate to others what strategy to adopt and <strong>whether</strong> to run a migration at all.</p>
<p><img src="/assets/images/taxonomy.png" alt="Taxonomy of technology migrations" /></p>
<figcaption class="caption">Taxonomy of technology migrations</figcaption>
<h3 id="migrations-in-a-competitive-market">Migrations in a competitive market?</h3>
<p>It is extremely rare to talk about “migrations” in a competitive market scenario. There are two good reasons for this. The first is that they are better known under a different name.</p>
<p>When we find ourselves in a <strong>competitive market</strong> (with multiple competing technology providers) <strong>on a large scale</strong> (like throughout a country or the world), I believe that migrations are largely described by the <a href="https://en.wikipedia.org/wiki/Technology_adoption_life_cycle">technology adoption life cycle</a>. Yeah, exactly that one made famous by startups trying to show to investors how the world is going to react to their innovative product. Considering that the diffusion model it originated from has been used to track the purchase patterns of hybrid seed corn by farmers, we are definitely speaking about migrating from an old technology to a new one in broad terms An example closer to our times is the adoption of smartphones in the mobile phones market.</p>
<p><img src="/assets/images/technology_adoption_life_cycle.png" alt="Techonology adoption life cycle" /></p>
<figcaption class="caption">Techonology adoption life cycle</figcaption>
<p>Although these are not the kinds of migrations we are likely going to be running in our career as software engineers (usually the players of this game are companies, legislators or communities), they are a fascinating domain that can be reasoned about in terms of economics, demography, networks, game theory and anthropology. I wonder if anyone analyzed the migration to Python 3 this way ;)</p>
<p>I am not going to cover the technology adoption life cycle in detail, as there is already plenty of literature about it. If you are interested, I recommend you start reading from the <a href="https://en.wikipedia.org/wiki/Technology_adoption_life_cycle">Wikipedia page</a>.</p>
<p>Next is a <strong>competitive market on a small scale</strong> (like a company). I call this one the <strong>No-Migrations Quadrant</strong> because of the second reason why we don’t speak about “migrations” in competitive markets: <strong>we shouldn’t</strong>.</p>
<p>I believe that in such situations, instead of <strong>wasting</strong> time running several little migrations every time you gain a new user or use case, you should try to convince the people in charge that your technology is the best among all the ones available and that your team should have a <strong>monopoly</strong> over that technical area. The smaller the scale, the more forcing functions (e.g. management decisions) are effective as shortcuts to complete (or at least very large) adoption.</p>
<p>An example of one such migration that shouldn’t happen would be migrating the teams in your group from some in-house JavaScript framework to React, while another group is in the process of moving to Angular.</p>
<h3 id="migrations-under-monopoly">Migrations under monopoly</h3>
<p>Operating under a monopoly, your team would have <strong>agency over what technology to adopt</strong> and <strong>how and when</strong> it should be adopted, and, at the same time, <strong>supply</strong> such technology. This doesn’t mean that your team should necessarily build such technology. At the time of writing, the trend in the industry is for internal teams to offer infrastructure and platforms by assembling and gluing open-source and/or commercially available solutions (e.g. a computing platform based on Kubernetes and AWS EC2).</p>
<p>What kind of monopoly would be best for your company to adopt is a whole different topic altogether. My default suggestion is to go for a <strong>“benevolent” monopoly</strong>, where your team is responsible for providing a <strong>“preferred” technology</strong>, but other teams are allowed to adopt something else if they can properly justify why that is a better fit to their use case. It might help to avoid the stagnation of innovation typical of monopolies, but only if your team is open to recognizing and taking ownership of new successful technologies that others may end up using. You’ll also have to be comfortable with some degree of chaos, of course.</p>
<h4 id="diy-migrations">DIY migrations</h4>
<p>Moving on to the two monopolistic quadrants, things should start to look more familiar to most engineers in the crowd. When we talk about monopolies, scale is usually much smaller, so I tend to compare it to the <strong>capacity of the team</strong>.</p>
<p>In the scenario of a monopoly on a <strong>small scale</strong>, your team could migrate all use cases <strong>by itself</strong> in a reasonable time. And <strong>it should</strong>. For this reason, I call the ones in this category <strong>DIY migrations</strong>. They are relatively easy to pull off even with minimal instructions:</p>
<ul>
<li>You migrate a first use case while perfecting your technology.</li>
<li>You migrate a second one (or more) to make sure that everything works.</li>
<li>You migrate all the other use cases (perhaps with the help of some automation).</li>
</ul>
<p>A good example of this one would be to upgrade all services in your startup from Python 2 to Python 3.</p>
<p><img src="/assets/images/DIY_migrations.png" alt="DIY migration" /></p>
<figcaption class="caption">DIY migration</figcaption>
<p>When I suggest such an approach, new acolytes of migrations often look at me with wide eyes of disbelief. Changing other people’s code, deploying alien services, performing repetitive tasks: madness. You may well feel the same way; this doesn’t really sound like fun. It has one pretty damn good upside, though: <strong>little to no coordination required</strong>.</p>
<p>Coordinating with users, tracking down owners of legacy systems, persuading them to work on your migration instead of their priorities (or other migrations!), enabling and helping them to perform the migration, making sure everyone sticks to the deadline. Migrations rarely finish (or at least their length is highly unpredictable), because of the tremendous hidden cost of coordination. So rejoice: a little DIY can save you from the pain of herding human beings toward a not-so-shared goal!</p>
<h4 id="large-scale-migrations">Large-scale migrations</h4>
<p>The last kind of migration is the one most professional software engineers are interested in: <strong>migrations under monopoly on a large scale</strong> (usually a medium-sized to large company). In terms of team capacity, this means that your team would <strong>not</strong> be able to perform the migration by itself in a reasonable time. This means that some degree of coordination with users is necessary.</p>
<p>If you are struggling to understand whether your case falls in the DIY migrations’ category or in the large-scale migrations’ one, you are probably looking at a DIY migration. Like in the movies, you will recognize a large-scale migration as soon as you see it.</p>
<p>Inspired by the <a href="https://en.wikipedia.org/wiki/Technology_adoption_life_cycle">technology adoption life cycle</a>, I came up with a personal psychographic representation of the users’ distribution over time for an “ideal” migration under monopoly (at least for my definition of “ideal”). I did it mostly for fun, I must admit. But perhaps this curve will make it easier for me to describe, and for you to remember, what happens during each phase of a migration.</p>
<p>Please note that the Y axis represents the number of users who migrated at a given time, not the total number of users who already migrated (which amounts instead to the area below the chart); this works exactly the same of the technology adoption life cycle chart.</p>
<p><img src="/assets/images/migrations_under_monopoly.png" alt="Migration under monopoly" /></p>
<figcaption class="caption">Migration under monopoly</figcaption>
<p>If you are interested in this curve and in how to successfully pull off one of these migrations, follow me to the next post of the series.</p>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/mum-preparations/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">Migrations under monopoly: Preparations</figcaption>
</a>
</div>This is the first installment in a series of blog posts on the topic of technology migrations. If you want to start from the very beginning, you can find an overview of the series and an index of the posts here.The Technology Migrations series2020-10-18T20:00:00+00:002020-10-18T20:00:00+00:00https://poros.github.io/technology-migrations-series<h2 id="the-technology-migrations-series">The Technology Migrations series</h2>
<p><strong>Migrations are a messy business.</strong> They always run far behind schedule, and it is actually quite rare for them to end at all. They are hard to justify in terms of return on investment. They have a bad reputation among both users and management. They are emotionally draining. Yet they are the way things move forward, technologically speaking at least.</p>
<p>Having been working for my entire career (so far) on internal teams focused on infrastructure or platforms, I end up thinking about migrations a lot. I have built my own little taxonomy of technology migrations, I have come up with my personal recipe to pull them off, and I have initiated engineers in the craft of running them. And those are the topics and the intent of this series of blog posts.</p>
<p>The first post goes over the taxonomy of migrations and how to approach them based on the category they belong to. But you can start from the second one if you are only interested in how to run the most common type. In case you have read the entire thing already or you don’t have time for long reads, you can find a summary checklist to follow during your migration in the very last post.</p>
<ol>
<li><a href="https://poros.github.io/taxonomy-of-migrations/">A taxonomy of migrations</a></li>
<li><a href="https://poros.github.io/mum-preparations/">Migrations under monopoly: Preparations</a></li>
<li><a href="https://poros.github.io/mum-alpha/">Migrations under monopoly: Alpha</a></li>
<li><a href="https://poros.github.io/mum-beta/">Migrations under monopoly: Beta</a></li>
<li><a href="https://poros.github.io/mum-automation/">Migrations under monopoly: Automation</a></li>
<li><a href="https://poros.github.io/mum-nudges/">Migrations under monopoly: Nudges</a></li>
<li><a href="https://poros.github.io/mum-the-fat-tail/">Migrations under monopoly: The fat tail</a></li>
<li><a href="https://poros.github.io/mum-deprecation/">Migrations under monopoly: Deprecation</a></li>
<li><a href="https://poros.github.io/migration-checklist/">The migration checklist</a></li>
</ol>
<div align="center">
<a class="next-arrow" href="https://poros.github.io/taxonomy-of-migrations/">
<img style="max-width:5%" src="/assets/images/next_arrow.png" alt="Next" />
<b><figcaption class="caption">Next</figcaption></b>
<figcaption class="caption">A taxonomy of migrations</figcaption>
</a>
</div>The Technology Migrations series