I'm not a programmer but .. why bother asking you what length it's supposed to be if you're going to strlen() it anyway? Sure you might need to allocate resources if this was carrying a proper payload but here we're just talking about an index. The spec not specifying the max length of the payload seems to be the problem??
The maximum length is specified, 65535 bytes, because that is the largest number that fits into the length field. If you have a variable length field you always have to specify the length, otherwise you will never know the actual length and are for example unable to read the next field because you don't know where it starts. There is no real next field in the case of the heart beat packet, only padding, but you still have to figure out the length in order to know what to return.
You have the option to specify the length implicitly by using a terminator like a zero byte in a zero delimited string but this is sometimes a bad idea. First you have to scan the entire field just to figure out its length while with an explicitly stored length before the actual data you can just read the length and for example skip to the next field without ever looking at the field. Forgetting to properly terminate the field will of course get you in serious trouble, too. And last but not least using a terminator only works if you have an used symbol (or are willing to perform escaping). In the case of the heartbeat extension there are no restrictions on the content and you can put in arbitrary binary data and therefore something simple like a zero terminator does not work.
Using UDP over IPv4 isn't the max payload size for a packet 65,507 bytes? That would mean that the max length that can be specified is already going to be a potential buffer overflow. What happens if you use specify a zero length? Seems that if the protocol were being defined loosely that should allow for "any length of field that fits within the transport packet" and so allow for larger datagram packets (a la Jumbogram).
Is there a trend to leave definitions of these protocols as loose as possible (eg not specifying the response field in the heartbeat as a double/long or whatever) so as to allow hacking it to serve some other use later?
Guess I'm going to have to look at the DTLS protocol definitions ...
TLS is usually used on top of TCP and therefore UDP limits do not apply. I have no idea how implementations on top of UDP or other datagram protocols deal with that issue. For a payload length of zero I would expect to just get back a payload of length zero, but I am be no means a network or even TLS expert and the standard may as well disallow this. And I am to lazy to look it up.
It is probably a really bad idea to make your protocol depend on details of an underlying protocol layer. Just imagine sending something over Ethernet and receiving it at the other end over WLAN where both protocols may disagree on the maximum packet size. You usually want your standard as precise as possible without imposing unnecessary and artificial constraints. Being allowed to stuff undefined amounts of data into a heart beat packet just to ease abusing the protocol really seems to call for trouble.
