MP4 is a “container format” designed for having multiple media streams (typically video, audio, and subtitles) in one file. Despite the name, it’s not the successor to MP3. The fact that a file is an MP4 doesn’t tell you what compression was used to compress whatever audio (or other) streams might happen to be in the file. You can make an MP4 file with a single FLAC audio stream in it. Or an MP4 file with a single MP3 or OGV or Speex stream in it. And the size of an MP4 can vary wildly based on what codecs were used to compress whatever streams might be in it. An .m4a file is just an MP4 file with a different extension intended to indicate that that particular MP4 file only has audio in it.

But anyway, to your question, the thing with lossless compression like FLAC is that it gives better fidelity at the cost of larger file sizes. Not as large as uncompressed audio files like WAVE files, but much larger typically than lossy-compressed formats like MP3, OGV, Speex, Opus, AAC, WMA, etc.

If you’re wanting to keep the same level of fidelity (that is, lossless), your options are basically (and being honest here, I got this list with like 30 seconds of googling) FLAC, ALAC, Ape, and WavePack. I’ve seen a couple of mentions that Ape may be a very tiny bit better than FLAC, but at the cost of much more computational complexity (e.g. more CPU usage). And WavePack might (I’m not sure) be a tiny bit better in some fairly narrow cases. ALAC seems mostly agreed to be inferior to FLAC in almost every way.

So, you could play with other lossless formats and see if you could shave a tiny bit off of your file sizes relative to FLAC, but the chances you’ll get a significant savings over FLAC without using a lossy format is are basically nil. And other lossless formats are likely to be more computationally intensive and less supported by hardware and software.

And this isn’t just because FLAC is not as good as it could be. Lossless compression is inherently going to give you bigger file sizes than lossy. FLAC has a certain amount of information it has to pack into a stream and, being lossless, it’s not allowed to strategically thow away part of that information to achieve a “balance” of fidelity versus file size the way lossy formats do. (Lossy formats mostly prioritize for discarding things that the human ear isn’t likely to hear like frequencies above or below the range of human hearing.) There’s only so much a compression algorithm can do to reduce file size within those constraints. And honestly, FLAC is really good at compressing efficiently within those constraints.

Now, all that said, I think most FLAC encoders offer the option of tuning the tradeoff between compression efficiency and CPU usage when decoding. If you wanted smaller FLAC files, again, the savings aren’t likely to be huge, but you could just up the compression level to 11 and let your CPU do a lot of extra work during playback just to save hard drive space. How to tune that compression-level parameter would depend what software you’re using to do the encoding. FFMPEG, for instance, has a compression level parameter.