You don’t even have to do that, you can just go to Settings.

Some users are reporting that a new banner is appearing at the top of the Edge browser that says the app “now launches when you sign into Windows, so it’s ready when you want to browse. Change this anytime in Settings.”

Windows Central can confirm that as of the latest Microsoft Edge Beta build, this banner does appear at the top of the interface, and unless you select the “No thanks” button, the browser will automatically appear every time you start up your PC. The good news is that you can disable this, but the bad news is it appears to be opt-out rather than opt-in.