You can use SDFs (signed distance functions) to express your objects in a mathematical form. It has its pros and cons. One of the bigger cons is that most tools use the polygonal approach, so you'd have to build a lot from scratch.
Lots of SDF examples in ShaderToy[2], and many introductory videos on YouTube (I specially recommend Inigo Quilez's channel[2] --co-creator of [1]--, and The Art of Code [3])
It's completely doable, but most modeling software is built to produce triagonal meshes, and most graphics cards are optimized to render triagonal meshes.
