Skip to content

Commit

Permalink
Fix up projection bias calculation (#1059)
Browse files Browse the repository at this point in the history
### Description

This PR fixes which attribute the tensor containing the bias for the up
projection is assigned to.

### Motivation and Context

Previously, the calculated bias was incorrectly being assigned to the
down projection's bias instead of the up projection's bias.
  • Loading branch information
kunal-vaishnavi authored Nov 12, 2024
1 parent cc4577e commit 83ddc3d
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion src/python/py/models/quantized_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -333,7 +333,7 @@ def __init__(self, quant_type, input_path, bits, group_size, q_size, kv_size, in
# model.layers.layer_id.mlp.gate_up_proj.bias
# model.layers.layer_id.mlp.dense_h_to_4h.bias
module.mlp.gate_proj.bias = tensor[: intermediate_size]
module.mlp.down_proj.bias = tensor[intermediate_size: ]
module.mlp.up_proj.bias = tensor[intermediate_size: ]
else:
raise NotImplementedError(f"{name} in your quantized model is not recognized.")

Expand Down

0 comments on commit 83ddc3d

Please sign in to comment.