Abstract: The whirling frequency and the critical speed of a rotating cantilever Rayleigh shaft are studied in this paper, based on the Rayleigh beam model, and the motion equation of the rotating cantilever Rayleigh shaft is derived, and discretized by the Galerkin method. During the Galerkin process, the modal shape functions of the non-rotating Euler-Bernoulli beam and the rotating Rayleigh beam with clamped-free boundary conditions are selected as the trial functions to obtain the whirling frequencies and the critical speeds. Both solutions are illustrated by numerical examples, the convergence of solutions is tested, and the results are compared with the classical solution obtained analytically. It is shown that using the modal shape of the non-rotating Euler-Bernoulli beam to obtain the approximation is usually far easier and faster than using the modal shape of the rotating Rayleigh beam. Therefore, it is preferred for the dynamic solution of the rotating cantilever shaft.