This is a good introduction on a conceptual level.
I think a large contributor to the problem is story-oriented development, where all that matters in the sprint is "getting it done" and not looking at the broader context.
To make unrepresentable states practical, Scott Wlaschin has an excellent write-up here (0). His book (plugged in that article) is also excellent.
> I think a large contributor to the problem is story-oriented development, where all that matters in the sprint is "getting it done" and not looking at the broader context.
I think you have a point here. This design offers much better safety, comparable to "parsing instead of validating". But it requires up-front design. And that is indeed "verboten" in modern software development management style.
Why is it "verboten"? I think it has to do with two fundamental concepts of scrum et. al.:
1. Stories that are "ready" just need to be "implemented". The implication is that the developer does not design and everything is orthogonal. There are no interdependencies, maintenance effort or non-functional requirements.
2. A story that is "ready" focuses solely on the desired outcome for some selected examples. There is no generalization of the examples and consequently little to no abstraction.
I think these issues stem from the fact that Scrum et. al. are intrinsically tools for managers to isolate them from the complexities of software engineering. Every metric of scrum, for instance, like "progress" or "definition of ready" is essentially empty of meaning for software engineering.
I like how you've said it here, but one thing that scrum doesn't have in it is relief from your professional duty as an engineer.
If a system needs to be designed in a particular way, do so. That's how long it takes and that's why in planning you discuss how it will be designed.
The design of how you're going to build the system is taken care of before the task is split into easily digestible bits that meet a definition of ready.
If you need to build it before you know how to build it, scrum allows for research spikes into this. A focused, timeboxed interval that so you have the ability to estimate the actual difficulty of work to be completed.
I get the impression that anything management touches gets turned (one might say corrupted) into a tool for detaching from engineering details. That's not what sole was meant to be about, and it's not what waterfall was about, but it happens anyway.
Details will always matter, but management's paycheck depends on not understanding that.
> I think a large contributor to the problem is story-oriented development, where all that matters in the sprint is "getting it done" and not looking at the broader context.
I think that's exactly backwards. This kind of overcomplicated representation usually happens because people put too much effort into designing their representations up front. If you follow story-oriented development and only implement the parts they actually need to get the current task done, you never end up with these wasteful extra states because you never actually needed them. But people think that planning before coding is somehow virtuous, and then they're tied to following those plans.
- A team spends too much time over-engineering a representation for something that could be more easily maintained using a simple model
- A team spends too little time considering the edge cases with a representation because they feel pressure to deliver the feature within a short space of time
And they often happen at the same time, too. I've seen situations where developers over-plan a system in advance, then spend ages hacking smaller features and changes into it to meet quick turnaround times, instead of taking proper step back and being aware of when their original design needs to be revised.
I assure you that I have wasted enough time fixing story-oriented development with major refactorings, because some edge cases weren't possible to be easily extended in the existing code.
Also have experience fixing story-oriented development with dirty workarounds, because major refactorings were also required, but not desired by whom was paying for the stories.
Both ways I didn't care, it was money on the bank anyway.
the flip side of this is that sometimes your initial designs need to be expanded to account for something new and that's hard, leading to tech debt and hacks.
personally I think that's a better trade-off than implementing something complicated up-front that you don't know will work and ending up with flexibility in the wrong places, leading to tech debt and hacks.
The entire point of sum types is that they be statically checked. Without static typing, they don't seem really useful: Erlang doesn't have sum types, and it doesn't really have a use for them until it gets a type system which can leverage them. Instead it models "sum types" as tagged tuples e.g.
{ok, Ok} | {err, Err}
however Erlang has very good pattern matching. Python… doesn't.
There's a PEP but I stopped following it because the discussion was a mess. And it's apparently now split into 3 different PEP, I don't know whether that's an improvement or not though.
Furthermore Python's dislike of HOFs means you can't really do "monadic" processing as you'd do in, say, smalltalk where your "variants" would really be subtypes with cool higher-order messages. So you're mostly just adding indirections.
I'm not sure you would consider this a good way, but one can implement (unchecked) sum and product types in any language with lexical closures via Scott encoding[1]:
# data Pair a b = Pair a b
def pair(x, y):
return lambda f: f(x, y)
# data Either a b = Left a | Right b
def left(x):
return lambda l, r: l(x)
def right(y):
return lambda l, r: r(y)
v0 = pair(2, 3)
v1 = left(7)
v2 = right(9)
# case v0 of { Pair x y -> x + y }
print(v0(lambda x, y: x + y))
# case v1 of { Left x -> x + 1; Right y -> y * 2 }
print(v1(lambda x: x + 1, lambda y: y * 2))
It doesn't sound to me like the kind of thing that any particular WoW is going to fix. As if it would suddenly naturally dawn on someone to do this given enough time. It's just lack of knowledge and/or discipline.
I think a large contributor to the problem is story-oriented development, where all that matters in the sprint is "getting it done" and not looking at the broader context.
To make unrepresentable states practical, Scott Wlaschin has an excellent write-up here (0). His book (plugged in that article) is also excellent.
[0] https://fsharpforfunandprofit.com/posts/designing-with-types...