You can connect two external 4k monitors at 60hz using just one thunderbolt port of the Macbook Air with a thunderbolt 4 dock. It will not work with a thunderbolt 3 or lower dock. I’ve personally tested the Amazon Basics Thunderbolt4/USB4 Pro Docking Station and it works perfectly.
In order to have two 4k monitors at 60hz, you’ll need to use both of the thunderbot ports on this docking station. It you use the hdmi port and one thunderbolt port, the monitor connected to the thunderbolt port will only be 30hz. I have tested the WARRKY USB C to HDMI Cable 4K 60Hz[Anti-Interference Gold-Plated Plugs] Aluminum Type-C to HDMI 2.0 Cord and it works. Using two of these cables, I was able to get two 4k monitors at 60hz. This leaves my second thunderbolt port on my mac free to be used for something else.