I have three different kinds of keyboards. Regular, Alice and split ergo with columnar stagger. It always takes a little while to get back to speed when I switch boards. But I think that the layouts are different enough so that a distinct muscle memory forms for each one. I guess that the most important thing is that you should be fairly proficient with touch typing on a regular board before trying a different layout. If you are still in the phase of learning, I imagine that it gets really frustrating.