Did you check with your fab house if they are OK with oval through holes?
0.6 mm milling diameter is not for the faint of heart
Can you post a link to the datasheet for the connector please?
If I was you I would take artistic liberty and mill it all with the same diameter.
Is this for production or hobby?
In case of hobby I would do 0.8 mm milling.
As for the 3D viewer display glitch, don't worry, it shouldn't affect the outcome.
I would be wary on those 'sideway' slots though, their position in your footprint isn't centered on the large one as it's given in the drawing..