This is notoriously difficult to quantify even in the simplest of examples. How do you even express goals here? What if the software is targeting a variety of platforms?
> peak memory consumption
Is this software running on a server or a users computer? What if it’s cheaper to buy more ram? How do you even measure this accurately when garbage collectors are so prevalent and there’s performance wins from using more virtual memory?
> Correct behaviour for injected faults
A standard set of faults? What about combinations of faults?
This is notoriously difficult to quantify even in the simplest of examples. How do you even express goals here? What if the software is targeting a variety of platforms?
> peak memory consumption
Is this software running on a server or a users computer? What if it’s cheaper to buy more ram? How do you even measure this accurately when garbage collectors are so prevalent and there’s performance wins from using more virtual memory?
> Correct behaviour for injected faults
A standard set of faults? What about combinations of faults?
I can go on….