For the record, we aren't endorsing CryptoCat or any of the other tools that got 7/7: ChatSecure, TextSecure, Signal/RedPhone, SilentText, SilentPhone, or the ones that are close like Pidgin+OTR, Subrosa, Surespot, Telegram, Threema or iMessage.
Getting those scores is a sign that those projects are taking the right approach. Lots of codebases have horrific bugs, including OpenSSL, older versions of the SSL and TLS protocol itself. We believe that focusing the community on the task of moving the best projects forward is more constructive than
Testing the tools that are scoring highest for usability, and doing deeper examinations of their designs and codebases, is going to be a future component of this campaign.
That sounds like a good start - but I still think the way you're presenting this data is somewhat misleading. I agree that many codebases have horrific bugs, and ultimately exploits will eventually come to light in existing applications.. However instead of listing many different available applications, why not just list what works for "right now"?
For instance: "Encrypted so the provider can’t read it?" on Skype's messaging just isn't true. If a subpoena was issued to Microsoft for conversation data, it'd be available.
I think it'd be better to stand up what does things right, and for everything else say why you're not listing them as effective. Approach is all well and good, but to an anonymous journalist/source being pursued a groups - approach over execution could be the difference between freedom and imprisonment (or worse).
Sorry, I tried to reply here yesterday but was rate-limited out of the conversation :/
Our aim with this project is to not give advice about what works "right now", because we aren't convinced there are any secure messaging options right now, especially when the usability dimensions of security are taken seriously.
Instead, what we're trying to do is articulate the things that both large companies and open source projects need to be doing to move in the right direction.
Since this is phase 1 of a multi-part campaign, we're going to take a closer look at the usability and further security properties of tools that are doing well on the Scorecard in subsequent phases.
On Skype, before launch it wasn't clear clear to us whether the NSA's reported Skype intercept capability came from breaking or having Microsoft backdoor the crypto (which would mean they loose the second checkmark) or by having Microsoft hand out a false public key for the other party (which is possible due to the lack of a check mark in the third column). We have an ongoing conversation with Microsoft about this and are reviewing Skype's ratings at the moment.
Oh I see, so the intended outcome is to raise awareness with the general public and shed light on issues with existing software they use in hopes that the powers at be take notice?
You may not intend to endorse CryptoCat, but it sure looks like it.
You should provide a simplified rating or some end user friendly explaination that is hard to ignore. It's confusing and misleading to say something is secure but there's no way to e.g. verify keys.
Getting those scores is a sign that those projects are taking the right approach. Lots of codebases have horrific bugs, including OpenSSL, older versions of the SSL and TLS protocol itself. We believe that focusing the community on the task of moving the best projects forward is more constructive than
Testing the tools that are scoring highest for usability, and doing deeper examinations of their designs and codebases, is going to be a future component of this campaign.