v1.5.5
What's Changed
- Deprecate runtime special tokens for tekkenizer. by @juliendenize in https://github.com/mistralai/mistral-common/pull/79
- Change default tekken special token policy by @juliendenize in https://github.com/mistralai/mistral-common/pull/80
New Contributors
- @juliendenize made their first contribution in https://github.com/mistralai/mistral-common/pull/79
Full Changelog: https://github.com/mistralai/mistral-common/compare/v1.5.4...v1.5.5