Question about linear quantization

[frankinwi]

I figure out the procedure of linear quantization and reproduce the experiments,

1. Search the quantization strategy on the imagenet100 dataset.
2. Finetune the model on the whole imagenet dataset with the strategy obtained from step 1.

It seems like the final accuracy of the quantized model is more dependent on the fine-tuning.
Another question is why the bit reduction process starts from the last layer as the _final_action_wall function shows.

[87Candy]

Could I ask ,when linear_quantization', default=True,some error like the following will appear

Have you miss these errors?

[frankinwi]

@87Candy I have not encountered this error.
self._build_state_embedding() function is used to build the ten-dimensional feature vector as the paper section 3.1 shows, you can check it.

[87Candy]

There are two methods, one is the K-means quantification method, the other is the linear quantification, I would like to ask, when you ran through the linear quantization, what changes have been made to the entire project file?
Thanks for your solution.

[frankinwi]

@87Candy The error may be caused by data = torch.zeros(1, 3, H, W).cuda() in measure_model function. You can change the batch size.

[87Candy]

@87Candy The error may be caused by data = torch.zeros(1, 3, H, W).cuda() in measure_model function. You can change the batch size.

some another question maybe encouter,could I communicate with you,one more time?

[alan303138]

How to covert mobilenet v2 in to qmobilenetv2?
qmobilenetv2 seems using QConv2d and QLinear, then how can I calibrate the bits into mobilenetv2?

[frankinwi]

@alan303138 See

haq/lib/env/linear_quantize_env.py

Line 115 in 8228d12

self.model = calibrate(self.model, self.train_loader)

[alan303138]

@alan303138 See

haq/lib/env/linear_quantize_env.py

Line 115 in 8228d12

self.model = calibrate(self.model, self.train_loader)

Thank you for your reply,
But according to the pre-training file they provided : mobilenetv2-150.pth.tar ,it seems that there is no inheritance relationship for QModule, because it is implemented in models/mobilnetv2, so do I need to inherit the model or is there something I missed?

And I also used the QConv2d and QLinear provided by them to make a pretrained qmobilnetv2, but I am not sure if this is correct, because it is not usually necessary to use fp32 training and then convert it to a quantized model?(Like quantization aware training)

haq/lib/utils/quantize_utils.py

Line 454 in 8228d12

if isinstance(module, QModule):

# If my current model is mobilenetv2 it will not do calibrate,because the implementation inside is nn.Conv2d, nn.Linear

haq/lib/utils/quantize_utils.py

Line 455 in 8228d12

module.set_calibrate(calibrate=True)

[frankinwi]

@alan303138

1. Modify the strategy = [[8,-1], [8,8], [8,8], [8,8],...., [8,8]] and use run_linear_quantize_finetune.sh to obtain a W8A8 quantized mobilnetv2 model.
   

   haq/finetune.py

   Line 316 in 8228d12

   strategy = [[8, -1], [7, 7], [5, 6], [4, 6], [5, 6], [5, 7], [5, 6], [7, 4], [4, 6], [4, 6], [7, 7], [5, 6], [4, 6], [7, 3], [5, 7], [4, 7], [7, 3], [5, 7], [4, 7], [7, 7], [4, 7], [4, 7], [6, 4], [6, 7], [4, 7], [7, 4], [6, 7], [5, 7], [7, 4], [6, 7], [5, 7], [7, 4], [6, 7], [6, 7], [6, 4], [5, 7], [6, 7], [6, 4], [5, 7], [6, 7], [7, 7], [4, 7], [7, 7], [7, 7], [4, 7], [7, 7], [7, 7], [4, 7], [7, 7], [7, 7], [4, 7], [7, 7], [8, 8]]

2. Modify the path variable to the W8A8 quantized mobilnetv2 model obtained in step 1.
   

   haq/models/mobilenetv2.py

   Line 186 in 8228d12

   path = 'pretrained/imagenet/mobilenetv2-150.pth.tar'

3. Run the run_linear_quantize_search.sh to perform the RL-based bitwidth search process to obtain an optimal strategy. As the upper bound of the action space is 8-bit, you should use W8A8 quantized model as the baseline (i.e., step 1). That is why the float_bit and the max_bit in run_linear_quantize_search.sh script is 8-bit.
   

   haq/run/run_linear_quantize_search.sh

   Line 8 in 8228d12

   --float_bit 8 \

4. Modify the strategy = "the searched optimal strategy in step 3 " and use run_linear_quantize_finetune.sh to recover the accuracy of the mixed-precision quantized model.
   

   haq/finetune.py

   Line 316 in 8228d12

   strategy = [[8, -1], [7, 7], [5, 6], [4, 6], [5, 6], [5, 7], [5, 6], [7, 4], [4, 6], [4, 6], [7, 7], [5, 6], [4, 6], [7, 3], [5, 7], [4, 7], [7, 3], [5, 7], [4, 7], [7, 7], [4, 7], [4, 7], [6, 4], [6, 7], [4, 7], [7, 4], [6, 7], [5, 7], [7, 4], [6, 7], [5, 7], [7, 4], [6, 7], [6, 7], [6, 4], [5, 7], [6, 7], [6, 4], [5, 7], [6, 7], [7, 7], [4, 7], [7, 7], [7, 7], [4, 7], [7, 7], [7, 7], [4, 7], [7, 7], [7, 7], [4, 7], [7, 7], [8, 8]]

[alan303138]

@frankinwi
Thank you for the very detailed steps,still not sure
So I can't use mobilenetv2-150.pth.tar this model right?(Only for kmean quant?)
If I use --arch qmobilenetv2. The mobilenetv2 will implement using QConv2d and Qlinear not nn.Conv2d nn.Linear which implement in pretrained model:mobilenetv2-150.pth.tar.

haq/run/run_linear_quantize_finetune.sh

Line 3 in 8228d12

-a qmobilenetv2 \

mobilenetv2

haq/models/mobilenetv2.py

Line 169 in 8228d12

model = MobileNetV2(**kwargs)

qmobilenetv2

haq/models/mobilenetv2.py

Line 183 in 8228d12

model = MobileNetV2(conv_layer=QConv2d, num_classes=1000, **kwargs)

[frankinwi]

@alan303138

1. The QConv2d inherits from the QModule base class.
   

   haq/lib/utils/quantize_utils.py

   Line 363 in 8228d12

   class QConv2d(QModule):

   
   The construction function of QConv2d initializes the w_bit=-1, which will first initialize the self._w_bit = w_bit in QModule, i.e., self._w_bit = w_bit=-1
   

   haq/lib/utils/quantize_utils.py

   Line 366 in 8228d12

   w_bit=-1, a_bit=-1, half_wave=True):

2. When running the forward function of QConv2d, it will first call self._quantize_activation(inputs=inputs) then self._quantize_weight(weight=weight).
   

   haq/lib/utils/quantize_utils.py

   Line 395 in 8228d12

   inputs, weight, bias = self._quantize(inputs=inputs, weight=self.weight, bias=self.bias)

3. Take self._quantize_weight(weight=weight) as example, now self._w_bit = w_bit=-1, it will jump to line 315 and return weights without quantization.
   

   haq/lib/utils/quantize_utils.py

   Line 287 in 8228d12

   if self._quantized and self._w_bit > 0:

   
   

   haq/lib/utils/quantize_utils.py

   Line 315 in 8228d12

   else:

Putting them all together, if we do not use half-precision (fp16, see --half flag) and do not specify the w_bit and a_bit for each QConv2d and QLinear layer, the qmobilenetv2 will not be quantized.

According to run_pretrain.sh and pretrain.py, the pre-trained file mobiletv2-150.pth.tar seems to use fp16. Therefore, the mobiletv2-150.pth.tar file might be unsuitable for linear quantization. You can load the mobiletv2-150.pth.tar and insert some prints before

haq/models/mobilenetv2.py

Line 192 in 8228d12

return model

to check it out.