Convolutional neural networks (CNNs) have a large number of redundant parameters from the convolutional layer to the fully connected layer. Model pruning is an optimization technique used to reduce the memory consumption and floating-point operations (FLOPs) of deep convolutional neural network models. By removing “unimportant” weights from the model, it reduces the number of parameters and computation, while ensuring that the accuracy of the model is not affected as much as possible, achieving the best balance between parameter quantity and model performance. However, for the case of higher pruning rate, current channel pruning methods often use the norm the channel as the criterion of pruning, and lack the corresponding theoretical support. It is difficult to achieve the ideal pruning effect, which will lead to a significant decline in the accuracy of the compressed model. To solve this problem, this paper proposes a new channel pruning method based on sensitivity measurement, which uses second-order sensitivity as a criterion to measure the importance of channels. Through theoretical derivation, the traditional sensitivity calculation is extended from weight to channel, and it is proved that the sensitivity of the entire channel can be quantified by the sum of the weight’s sensitivity in the channel, and then the insensitive channel can be deleted to complete the channel pruning. Experiments on a variety of different CNNs architectures show that our paper can significantly improve the pruning rate while losing a small amount of accuracy. For example, with a slight loss of accuracy, FLOPs on the CIFAR-10 dataset were reduced by more than 60%. Additionally, on the ImageNet dataset, ResNet34-based pruning reduces FLOPs by 52.1% while losing only 0.23% accuracy.