Last Updated on March 9, 2015 by Mathew Diekhake
Hi Everyone,
I hear lots of questions about how the Insider Program works and what’s going on with Windows 10 development through the Windows 10 Community forum and on Twitter. It’s often hard there to answer questions for everyone to hear, or to do so with the kind of detail that I’d like. Instead, I’ll periodically post to the Insider Hub with info about top questions that I hear or a blog post like this one.
The first topic has been dominating my Twitter feed for the past 2 weeks: “I want the next build!” First off, I love hearing the genuine excitement that people have about getting new builds and trying out new features, it’s what we had hoped for when we started the Windows Insider Program. Thank you all for your enthusiasm.
Beyond the excitement for a new build, there are two points that are raised that I haven’t been able to respond to well in Twitter’s 140 characters:
- “I thought you said in January that builds will be coming faster from now on?”
- “Why can’t you just tell us when the next build will be? Can’t you be more transparent?”
With this post I hope to be able to shine some more light on both of these. We are certainly not trying to be opaque, but the answer is more complex than it may seem it should be, and we’ve admittedly taken too long to reply to these points and left people wondering.
Build cadence and where we are with “faster builds”
The short answer on this is really that we’ve probably been too conservative about pushing builds to the Fast ring for Windows Insiders.
We’re having lots of discussions right now about how we should tune for this going forward, and have already made some changes in how we’re working. The reality is that faster builds to you will include more bugs, and so far we’ve erred on the side of stability. In the process though we’ve not had as much distinction between Fast and Slow. In our internal rings, our Canary ring probably sees 2X-3X as many builds as OSG because we catch problems in Canary and don’t push to OSG.
Builds progress through flighting rings. If a build meets criteria, it moves on and the next ring receives it.
This is a new approach for us and we’re learning and evolving it as we go. On the outside it may have some similarity to past Preview programs or the old CTP programs from Vista and earlier, but there are myriad differences in both the approaches and underlying technologies. We also have some cool things yet to come. We’re debating right now about whether we should simply adjust the speed/risk balance of the Fast ring or whether we should create a new ring for people who really want the fastest pace possible (I had serious conversations last week with my team about a “Ludicrous Speed” ring, no joke.) As we make decisions on this we’ll communicate out to you so you know what to expect.
There will also be times when new functionality is coming into the builds where we need a little extra time to stabilize and polish before it reaches you. These feature integration windows will come at different times, which means we really cannot provide a predictable build schedule for when we expect builds to reach you. Sometimes they will be faster, and sometimes slower. This is why I’ve said on Twitter previously that we expect that you’ll get a build “roughly monthly” since we can typically expect that a build will come out every 30 days or so, though sometimes (as now) it may take a little longer.
Why it is a challenge to just tell a date for the next build
It is counter-intuitive, but doing this actually makes builds get published more slowly and have less fresh content than by leaving the date open ended. Why?
- If we announce a date, we’ll want to have a very high confidence of hitting it. It’s frustrating for you to hear a date and be let down if we miss it, and it’s frustrating and distracting for us too. Not only that, but it slows down our engineering since many of the same people who are scrambling after a missed date would otherwise have been making more forward progress on the product.
- Because we’d want that very high confidence we’d pick a date that was further out than if we were living on the edge. We’d give ourselves some time to deal with bugs and re-spin builds if we needed to.
- If we have a great build in hand, as often happens, leading up to the date we would hold on that build rather than ship it. We call this putting the build in ‘escrow’. Why not just ship it early? Well, some people get upset about the surprise, but also it sets expectations that sometimes we really mean a date and sometimes we don’t. We want people to know that when we say a date they can count on that date.
- In the worst case, if we’re chasing down a tough bug and run out of time, we may miss the date. This is of course way worse than being early. We’d have let down people who were counting on us to deliver on the date we said we would.
Let’s play that out hypothetically for the next build coming out. Today is 3/9 and we want to ensure we get a build out in March. If we communicated a target date, to be sure we could meet our commitment we’d likely pick a date like 3/26. It gives us time to stabilize and it’s on a Thursday (we usually like to avoid Mondays and Fridays.) Between now and then we’d still be getting new feature payloads, but we’d fork to a stabilization branch somewhere around 3/17 or so and only take selective changes. It’s easier to stabilize without a lot of additional new code, so we’d cherry pick key fixes. On 3/23 we’d have a candidate build, and we’d flight that out broadly within MS to make sure we could find any gotchas and meet our date with confidence. Hey, that doesn’t sound too bad does it? Except in the ‘worst case scenario’ where we miss the date and people are let down, it means a predictable date about once per month with kind of up to date code.
But now let’s talk about how we’re really trying to approach it. Today is 3/9 and we’ve not set a date for the next build. I have a build in hand that we produced on Friday. It was validated by our test automation, and will go out through our internal rings and get installed and used by thousands of people at Microsoft. It is the freshest code with all newest features and fixes. If it passes all of our evaluation criteria it could be in your hands late this week or early next week. That means that we could feasibly get multiple builds out in March rather than just one, and they’d have more up to date code than if we did it the other way. Yes, I know, that is pretty big talk considering it has been more than 40 days since our last build; and here I am talking about multiple builds per month. I’m sharing our aspirations and what we’re building towards, and we want to be working in that new way vs. the way we used to do it. Not having the constraint of a fixed public date for each build helps us get there faster. Here’s a real example though: We had a debate internally about whether we should announce a date for 9926 – the build we shipped the day after our 1/21 Windows 10 event. The choices we had:
- Set 1/23 as the announced date, knowing it would have been a build produced weeks before without many of the features we demoed.
- Set 2/15 as the announced date, giving ourselves the flexibility to have a good build in escrow.
- Don’t announce a date, use the ring promotion process, and go when ready.
We went with option 3 and it paid off. We got a much fresher build out, with more features and fixes, and we were able to ship on 1/23 as we’d aspired.
Keep the feedback and questions coming
I hope that this helped to give some insight into how we’re thinking about build cadence and how we’re responding to feedback. We really appreciate the time and thoughtfulness that Windows Insiders are putting into using the product and telling us what you think. Your feedback has helped us to shape the product in many ways, and we’ll do another post like this soon with some more of the inside scoop on how your feedback is used since that is another top question that we’re hearing from you.
Thank you,
g
Source: Frequency and predictability of builds for Windows Insiders