I think we need some sort of mlbench but for hardware ranking... I have no idea what these devices are capable of and neither do the vendors apparently.
MLCommons provides MLPerf which is a series of benchmarks anyone can run and compare results. For devices like the BeagleY-AI the "edge" category provides a series of various benchmarks to run covering things like object detection, speech recognition, and now smaller LLM's.
