Abstract: Railway ballast produces significant vibration and contact migration behaviour when transferring moving train loads, which induces breakage deterioration and non-uniform settlement deformation of the ballast bed. The research on the shear resistance of ballast materials is important for the dynamic behaviour and maintenance assessment of the ballast layer. To further investigate the shear mechanical properties of railway ballast, dilated polyhedron element was constructed based on Minkowski Sum theory in the discrete element method, and the irregular dilated polyhedron ballast particles were generated by three-dimensional Voronoi algorithm. A dilated polyhedron ballast model database that meets the railway ballast grading requirements was established by identifying and screening the particle size using the minimum projection method. The results for the quasi-static direct shear test and discrete element simulation of railway ballast under different normal stresses were compared to verify the rationality of contact parameters between the dilated polyhedron ballast elements. On this basis, the dynamic direct shear simulation of railway ballast under vibration fields was carried out to analyze the influence of vibration angle, amplitude and frequency on the shear performance of ballast material. The results show that the constructed dilated polyhedron ballast model can better reflect the strong occlusal and interlocking contact behaviours between irregular ballast materials. Meanwhile, the shear stress–shear strain and shear dilatancy (shrinkage) properties can also be well-modelled during the direct shear process. The shear strength of railway ballast corresponding to different normal stresses under regularization is better represented by a power function. Besides, the vibration field significantly reduces the shear resistance of ballast particles. When the vibration direction is the same as the shear direction (0° vibration angle), the shear strength of the ballast aggregates was significantly weakened. As the amplitude and frequency increase, the shear strength decreases significantly. This study demonstrates the need to pay more attention to the dynamic shear resistance of railway ballast in the performance assessment of the actual ballast bed.