This is exactly how Compilers are taught at the University of Maryland. The class CMSC430 (https://www.cs.umd.edu/class/fall2021/cmsc430/) actually starts off with a Scheme (limited subset of Racket) and gradually grows the language to include more features. The first class compiles just numbers to x86 code, followed by arithmetic operations for numbers, building up to higher level features like function calls, pattern matching, and so on. See the notes at: https://www.cs.umd.edu/class/fall2021/cmsc430/Notes.html

This style of building compilers is called the Nanopass style (https://legacy.cs.indiana.edu/~dyb/pubs/nano-jfp.pdf), making it much easier to teach.

Source: I was a TA for the earlier iteration of the class.