I like this mentality, truly, makes perfect sense. Check in often to make sure you’re building the right thing. Keep going until it’s done.
However, the client is going to want to know (A) how much it’s going to cost, and (B) how long it’s going to take. These are extremely reasonable questions in most cases/industries. To answer these questions with a shrug is a nonstarter. The client is working with a time budget and a financial budget, and they need to have some sense of the numbers.
If the Waterfall and Agile methods are opposite ends of a spectrum, somewhere in between is where I’ve found an acceptable middle ground for both developers and clients.
While I loathe the useless abstraction layer that is story points and while I generally agree with all of the points in articles like these, they almost never talk about the other side of the coin, which is the need for predictability.
Predictability is the oil that makes nearly every software business run efficiently and smoothly. It affects everything from software development to product roadmap to financial efficiency and profitability. Startups need to know if they have the ability to implement critical functionality before they're out of runway; big companies need to be able to coordinate product development, contracts, delivery dates, product launches across many disparate teams with interconnecting dependencies. Even deciding whether or not something is a roadmap priority requires some concept of how quickly you can implement it.
So yes, productivity theater, as it exists in many project management processes today is unnecessary overhead and wasted time/money. But unless you are id software in the late 90s—flush with cash and sitting on a couple of products that only you can bring to market—you can't rely on "it'll be done when it's done" or "you'll know when it's ready when you see it" and expect to remain competitive for long.
Not sure if this is intended to expand on my comment or to be a rebuttal to it, but to be clear: I am in full agreement with you.
My comment strictly refers to many of the articles[0] and comments[1] that push back on the modern incarnation of scrum/agile/whatever. Productivity theater in project management is net harmful to innovation and actual product development velocity, but my point is that we can't go to the other extreme and have no planning or any kind of measurement.
I despise SP too. I have come to the conclusion that all estimates should only ever given in terms of magnitude. "Some small number of hours/days/weeks/months/quarters/years". If product or leadership want more precision, break tasks into smaller self-contained tasks.
It's weird to me that people have emotional feelings about Story Points as a concept. It's just another way to measure how hard something might be. I think what people should really be annoyed with is when these measurements are used as some sort of productivity metric, or if the team spends too much time debating a particular measurement value and not enough time actually working on it.
My annoyance with story points is how it always seems to end up returning to “how many hours of work will it take”, even though the whole point of using story points is to get away from trying to predict how many hours of work something is.
That is, we estimate a certain set of tasks. For this two-week sprint, we're going to try to do a subset, and that subset adds up to 20 story points. After two weeks, how much did we actually get done? 7 story points. Next sprint we did better, we got 11 done. After a few months, we settle down to an average of 10 story points per two week sprint. Now we know how many hours something is (estimated to be) based on the story points.
Note well: This velocity is a function of the team. If the team composition changes, previously measurements of velocities are no longer valid.
In all my years of software development, story points never became an accurate predictor of time, even with consistent teams and process. The types of tasks we would be working on varied too much and were too novel to become predictable.
If we were working on one app and just adding features and fixing bugs, maybe it would converge to a consistent average. However, I have always worked on teams that have myriad projects, moving in and out of development, with constant support and interrupt driven work taking up a huge variable amount of time.
Management always gets frustrated when this doesn't materialize. If the instrument meant to keep management off your back doesn't do so, people will get frustrated with it.
In my experience, they're always just used as some abstraction over amounts of time, which doesn't seem particularly useful but also isn't objectionable. What's weird to me is how specific the patterns are sometimes; I've worked on more than one team that insists on only using Fibonacci numbers for story points, but also that anything as large as 8 should be broken up into separate tasks, which effectively means that they used the range 1-5 but forbid usage of 4. On one of those teams, during one planning meeting someone mused that they wished there were something they could use to represent "less than 1", and I suggested we try putting 0.5 story points into JIRA, which to everyone's surprised actually worked, so 0.5 became the only other allowed value.
I think SP or similar need to exist for it to be possible to make decisions and prioritise, but issues arise when part of the company, e.g. leadership or managers don't understand that SP are more like guesses with risk and probability involved and they will be disappointed when the end result won't be as accurate.
So naturally people will come to despise it because managers will want a number and to hold you to that number. A strict number can't be given, but intuitive guess which has certain probability of being in a certain range according to experience can be.
If you have two engineers and one consistently completes 10 points a sprint and the other only completes 2 points a sprint, does that not tell you something about the output of those engineers?
At best it may indicate that there's something worth looking into, but it doesn't tell you much about the actual productivity of the engineers. One engineer may be producing low quality output that requires a lot of re-work later, or they might be gaming the system by over-estimating work, or picking up lower priority work that was accidentally over-estimated in order to improve their numbers. They may be a domain expert in a particular system while the other developer is getting up to speed. One developer may be spending significantly more time mentoring or helping their team work better. They might be writing design documents or spending more time with customers. They might have been around longer and are regularly getting pulled into supporting things they worked on years ago, or getting asked for help from other teams who need their expertise.
Not without much more data. Is the 2 pt engineer the one senior who supports all the juniors and multiplies their effectiveness by getting them unstuck, or is the 2 pt engineer the one who always takes the hard (misestimated) stories, or maybe they are the CEO's nephew and they just suck. No way to know just from pts completed.
Does it tell something that couldn't be equally (or better) represented by not pretending that story points are time estimates rather than something abstract?
no, it tells you more about what sort of tasks they excel at and how story points are chosen. it's important not to extrapolate beyond what your measurement supports.
Mr 2 Points might be taking one for the team, doing a task that would cost Mrs 10 Points 3-5 points of productivity if they were saddled with it.
Low point stories that take a lot of time are often coordination tasks, and for people who are good at heads down programming, that can be their kryptonite.
It's also possible that Mr 2 Points is not getting fed stories that they could weave into the blocking points of their highest priority task effectively. He is spending a lot of time working on untracked tasks or sneakily working on stories halfway down the backlog. And they can't do it in the open because someone is engaging in Efficiency Theater: we are so far behind on some milestone that the optics of anyone working on anything except that milestone are terrible.
Nevermind that the next milestone needs them and we will be having this Death March repeat again in three months because of it.
Do you rewrite your auth system to modern standards or migrate your infra to k8s ? Some projects take months to complete and don’t deliver value until fully done, so it is valuable to spend some time to do some upfront planning to weigh your options
When I got my first EM job in 2011, implementing true agile was the dream. I became a zealot to every PM or business leader that came my way. I insisted that not only would we get more done, but we'd get better quality stuff. It was a beautiful dream.
I wouldn't say my idealism failed all at once, but through death by a thousand cuts. There are lots of scenarios where trying to bootstrap development into agile is simply more difficult without any clear gain...at least when it comes to principles around measuring progress.
In all my time in the industry, I've never met someone who lived true agile for more than a year that wasn't in a very small startup.
Hmm... doesn't really offer a suggestion for measuring progress. FWIW, in our dev team, we don't talk about "percent done," but say "percentage of unit tests that pass without mockups" and then "percentage of requirements covered by unit tests." The first measure is pretty easy to quantify, but the second is where ambiguity and guesswork creeps in.
Agile isn't a framework, it's just a set of principles. Scrum is a loose framework theoretically based off of those principles with some ground rules that can be broken depending on what works for your team. Same with Kanban.
The "waterfall" way of working still has some merit. Think first, then make a design, and a planning based on that, and then implement the fucking owl.
Unfortunately, a lot of unforeseen problems show up during the implementation phase, and people blame the waterfall model for not being flexible (agile) enough to address these.
Now, even more unfortunately, some people interpret agile as not needing to spend time upfront thinking and dive into the implementation straight away, creating an even bigger mess than the waterfall generation could have ever imagined.
> Unfortunately, a lot of unforeseen problems show up during the implementation phase, and people blame the waterfall model for not being flexible (agile) enough to address these.
my experience is that people spend half the time bikeshedding the obvious problems that waterfall would have identified in much less time, and then still hit the unforeseeable problems after implementation anyway. if they could have been foreseen, they would have been rolled into waterfall.
The last 2 companies I’ve worked at (one massive one tiny) both didn’t use any framework. Engineers/PMs/Managers worked together to figure out what to build and the engineers built it. The only “tool” was trust and it worked out fine. We did a short daily or weekly status update to let everyone know how things were going, but that’s about it. Some engineers liked to break out work into pieces and put that into a spreadsheet, but it was completely up to the builder to decide how to organize their work. When a notable piece of work got finished the engineer would demo it to the team/company so more people could see how it was going.
It’s not about like or dislike - it should have value in the context for which it’s designed. There are an unlimited amount of ways to do this. Validation in the context of the recipient of the value is the only thing that matters.
while i have only ever worked at places that do some form of agile development, sometimes i think about things that are just foundational elements. to be pedantic, they definitely work. they just, on their own, do not provide any utility. just a means to facilitate things to be constructed later on in development.
There's no point in measuring progress if you don't have a defined end point.
And if you have a defined endpoint, you will get the "is it far? Are we there yet?" question. For good reasons.
Maybe your software needs to hit at a certain point, or a lot of money will be losts (ask e-commerce folks about Black Friday). That means you'll need to try to course correct as soon as possible if you take the wrong road - "IDK, but we made progress in the last two weeks" isn't helping.
Maybe your software needs a marketing push. The buy for that is months ahead of when it happens. You better know if you can make that date.
Maybe other teams depend on you delivering working software at a certain point.
"We make progress every sprint" is a nice feeling. It doesn't help you in any of those situations. Especially large-scale efforts need some planning and estimation to work out.
We have all collectively forgotten that the main purpose of the burndown chart was to teach upper management about the wages of scope creep and vague requirements. They've taken our paddle and are spanking us with it.
I like this mentality, truly, makes perfect sense. Check in often to make sure you’re building the right thing. Keep going until it’s done.
However, the client is going to want to know (A) how much it’s going to cost, and (B) how long it’s going to take. These are extremely reasonable questions in most cases/industries. To answer these questions with a shrug is a nonstarter. The client is working with a time budget and a financial budget, and they need to have some sense of the numbers.
If the Waterfall and Agile methods are opposite ends of a spectrum, somewhere in between is where I’ve found an acceptable middle ground for both developers and clients.
While I loathe the useless abstraction layer that is story points and while I generally agree with all of the points in articles like these, they almost never talk about the other side of the coin, which is the need for predictability.
Predictability is the oil that makes nearly every software business run efficiently and smoothly. It affects everything from software development to product roadmap to financial efficiency and profitability. Startups need to know if they have the ability to implement critical functionality before they're out of runway; big companies need to be able to coordinate product development, contracts, delivery dates, product launches across many disparate teams with interconnecting dependencies. Even deciding whether or not something is a roadmap priority requires some concept of how quickly you can implement it.
So yes, productivity theater, as it exists in many project management processes today is unnecessary overhead and wasted time/money. But unless you are id software in the late 90s—flush with cash and sitting on a couple of products that only you can bring to market—you can't rely on "it'll be done when it's done" or "you'll know when it's ready when you see it" and expect to remain competitive for long.
EDIT: Mobile typos.
Agile is not "it's done when it's done".
Agile is "it makes sense every day of the week, we should talk often".
Not sure if this is intended to expand on my comment or to be a rebuttal to it, but to be clear: I am in full agreement with you.
My comment strictly refers to many of the articles[0] and comments[1] that push back on the modern incarnation of scrum/agile/whatever. Productivity theater in project management is net harmful to innovation and actual product development velocity, but my point is that we can't go to the other extreme and have no planning or any kind of measurement.
[0] https://rethinkingsoftware.substack.com/p/why-scrum-is-stres...
[1] https://news.ycombinator.com/item?id=41545101
Agile's one of the best things to happen to software in a way: it's good to know your competitors are stuck in ceremonies, arguing over story points.
Looks like you’re talking about Scrum. Ceremonies and story points aren’t in any way mandated by the Agile manifesto.
There is no "ceremonies" nor "story points" in the Scrum Guide. Perhaps you are referring to "Fake Scrum".
One of my favorite responses to criticism was Dan Abramov telling people if Redux doesn't work for them, don't use it!
Keep what works for you, leave what doesn't.
I despise SP too. I have come to the conclusion that all estimates should only ever given in terms of magnitude. "Some small number of hours/days/weeks/months/quarters/years". If product or leadership want more precision, break tasks into smaller self-contained tasks.
It's weird to me that people have emotional feelings about Story Points as a concept. It's just another way to measure how hard something might be. I think what people should really be annoyed with is when these measurements are used as some sort of productivity metric, or if the team spends too much time debating a particular measurement value and not enough time actually working on it.
My annoyance with story points is how it always seems to end up returning to “how many hours of work will it take”, even though the whole point of using story points is to get away from trying to predict how many hours of work something is.
You don't predict that. You measure it.
That is, we estimate a certain set of tasks. For this two-week sprint, we're going to try to do a subset, and that subset adds up to 20 story points. After two weeks, how much did we actually get done? 7 story points. Next sprint we did better, we got 11 done. After a few months, we settle down to an average of 10 story points per two week sprint. Now we know how many hours something is (estimated to be) based on the story points.
Note well: This velocity is a function of the team. If the team composition changes, previously measurements of velocities are no longer valid.
In all my years of software development, story points never became an accurate predictor of time, even with consistent teams and process. The types of tasks we would be working on varied too much and were too novel to become predictable.
If we were working on one app and just adding features and fixing bugs, maybe it would converge to a consistent average. However, I have always worked on teams that have myriad projects, moving in and out of development, with constant support and interrupt driven work taking up a huge variable amount of time.
Management always gets frustrated when this doesn't materialize. If the instrument meant to keep management off your back doesn't do so, people will get frustrated with it.
In my experience, they're always just used as some abstraction over amounts of time, which doesn't seem particularly useful but also isn't objectionable. What's weird to me is how specific the patterns are sometimes; I've worked on more than one team that insists on only using Fibonacci numbers for story points, but also that anything as large as 8 should be broken up into separate tasks, which effectively means that they used the range 1-5 but forbid usage of 4. On one of those teams, during one planning meeting someone mused that they wished there were something they could use to represent "less than 1", and I suggested we try putting 0.5 story points into JIRA, which to everyone's surprised actually worked, so 0.5 became the only other allowed value.
I think SP or similar need to exist for it to be possible to make decisions and prioritise, but issues arise when part of the company, e.g. leadership or managers don't understand that SP are more like guesses with risk and probability involved and they will be disappointed when the end result won't be as accurate.
So naturally people will come to despise it because managers will want a number and to hold you to that number. A strict number can't be given, but intuitive guess which has certain probability of being in a certain range according to experience can be.
Just to play devils advocate:
If you have two engineers and one consistently completes 10 points a sprint and the other only completes 2 points a sprint, does that not tell you something about the output of those engineers?
At best it may indicate that there's something worth looking into, but it doesn't tell you much about the actual productivity of the engineers. One engineer may be producing low quality output that requires a lot of re-work later, or they might be gaming the system by over-estimating work, or picking up lower priority work that was accidentally over-estimated in order to improve their numbers. They may be a domain expert in a particular system while the other developer is getting up to speed. One developer may be spending significantly more time mentoring or helping their team work better. They might be writing design documents or spending more time with customers. They might have been around longer and are regularly getting pulled into supporting things they worked on years ago, or getting asked for help from other teams who need their expertise.
> At best it may indicate that there's something worth looking into
Or as I usually put it: Statistics/charts are for asking questions, not answering them.
Not without much more data. Is the 2 pt engineer the one senior who supports all the juniors and multiplies their effectiveness by getting them unstuck, or is the 2 pt engineer the one who always takes the hard (misestimated) stories, or maybe they are the CEO's nephew and they just suck. No way to know just from pts completed.
Does it tell something that couldn't be equally (or better) represented by not pretending that story points are time estimates rather than something abstract?
Not without more data.
no, it tells you more about what sort of tasks they excel at and how story points are chosen. it's important not to extrapolate beyond what your measurement supports.
Mr 2 Points might be taking one for the team, doing a task that would cost Mrs 10 Points 3-5 points of productivity if they were saddled with it.
Low point stories that take a lot of time are often coordination tasks, and for people who are good at heads down programming, that can be their kryptonite.
It's also possible that Mr 2 Points is not getting fed stories that they could weave into the blocking points of their highest priority task effectively. He is spending a lot of time working on untracked tasks or sneakily working on stories halfway down the backlog. And they can't do it in the open because someone is engaging in Efficiency Theater: we are so far behind on some milestone that the optics of anyone working on anything except that milestone are terrible.
Nevermind that the next milestone needs them and we will be having this Death March repeat again in three months because of it.
Do you rewrite your auth system to modern standards or migrate your infra to k8s ? Some projects take months to complete and don’t deliver value until fully done, so it is valuable to spend some time to do some upfront planning to weigh your options
When I got my first EM job in 2011, implementing true agile was the dream. I became a zealot to every PM or business leader that came my way. I insisted that not only would we get more done, but we'd get better quality stuff. It was a beautiful dream.
I wouldn't say my idealism failed all at once, but through death by a thousand cuts. There are lots of scenarios where trying to bootstrap development into agile is simply more difficult without any clear gain...at least when it comes to principles around measuring progress.
In all my time in the industry, I've never met someone who lived true agile for more than a year that wasn't in a very small startup.
Hmm... doesn't really offer a suggestion for measuring progress. FWIW, in our dev team, we don't talk about "percent done," but say "percentage of unit tests that pass without mockups" and then "percentage of requirements covered by unit tests." The first measure is pretty easy to quantify, but the second is where ambiguity and guesswork creeps in.
This was interesting to read as a more "junior" software dev. Aside from Agile, what are other common frameworks that people use, if any?
Agile isn't a framework, it's just a set of principles. Scrum is a loose framework theoretically based off of those principles with some ground rules that can be broken depending on what works for your team. Same with Kanban.
The "waterfall" way of working still has some merit. Think first, then make a design, and a planning based on that, and then implement the fucking owl.
Unfortunately, a lot of unforeseen problems show up during the implementation phase, and people blame the waterfall model for not being flexible (agile) enough to address these.
Now, even more unfortunately, some people interpret agile as not needing to spend time upfront thinking and dive into the implementation straight away, creating an even bigger mess than the waterfall generation could have ever imagined.
> Unfortunately, a lot of unforeseen problems show up during the implementation phase, and people blame the waterfall model for not being flexible (agile) enough to address these.
my experience is that people spend half the time bikeshedding the obvious problems that waterfall would have identified in much less time, and then still hit the unforeseeable problems after implementation anyway. if they could have been foreseen, they would have been rolled into waterfall.
The last 2 companies I’ve worked at (one massive one tiny) both didn’t use any framework. Engineers/PMs/Managers worked together to figure out what to build and the engineers built it. The only “tool” was trust and it worked out fine. We did a short daily or weekly status update to let everyone know how things were going, but that’s about it. Some engineers liked to break out work into pieces and put that into a spreadsheet, but it was completely up to the builder to decide how to organize their work. When a notable piece of work got finished the engineer would demo it to the team/company so more people could see how it was going.
Extreme Programming used to be a thing. Pair programming and all that... Not so much anymore.
Pair programming does not lend itself well to micromanagement.
there's kanban
I would like to hire you to come read this to my PM daily until he gets it.
your PM doesn't "get" how to build working software? Why are they the PM..?
Because it pays well? Anyway there are many PMs like this.
It’s not about like or dislike - it should have value in the context for which it’s designed. There are an unlimited amount of ways to do this. Validation in the context of the recipient of the value is the only thing that matters.
[dead]
while i have only ever worked at places that do some form of agile development, sometimes i think about things that are just foundational elements. to be pedantic, they definitely work. they just, on their own, do not provide any utility. just a means to facilitate things to be constructed later on in development.
in particular i think of Black Triangles https://rampantgames.com/blog/?p=7745 which has been discussed here previously.
There's no point in measuring progress if you don't have a defined end point.
And if you have a defined endpoint, you will get the "is it far? Are we there yet?" question. For good reasons.
Maybe your software needs to hit at a certain point, or a lot of money will be losts (ask e-commerce folks about Black Friday). That means you'll need to try to course correct as soon as possible if you take the wrong road - "IDK, but we made progress in the last two weeks" isn't helping.
Maybe your software needs a marketing push. The buy for that is months ahead of when it happens. You better know if you can make that date.
Maybe other teams depend on you delivering working software at a certain point.
"We make progress every sprint" is a nice feeling. It doesn't help you in any of those situations. Especially large-scale efforts need some planning and estimation to work out.
We have all collectively forgotten that the main purpose of the burndown chart was to teach upper management about the wages of scope creep and vague requirements. They've taken our paddle and are spanking us with it.