{"request":{"options":["mode=llm"],"pageUrl":"https://github.com/axolotl-ai-cloud/axolotl","api":"article","version":3},"objects":[{"date":"Wed, 13 May 2026 00:00:00 GMT","images":[],"author":"axolotl-ai-cloud","shareLinks":false,"estimatedDate":"Wed, 13 May 2026 00:00:00 GMT","icon":"https://github.com/fluidicon.png","diffbotUri":"article|3|374945869","siteName":"GitHub","type":"article","title":"axolotl-ai-cloud/axolotl","interactive_elements":[{"searchTemplate":"https://github.com/search?q=repo%3Aaxolotl-ai-cloud%2Faxolotl%20{searchTerms}&type=code&ref=opensearch","type":"search"}],"content":"GitHub - axolotl-ai-cloud/axolotl: Go ahead and axolotl questions · GitHub  [Skip to content ](https://github.com/axolotl-ai-cloud/axolotl/#start-of-content) \n\n## Navigation Menu Toggle navigation  [](https://github.com/)  [ Sign in  ](https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Faxolotl-ai-cloud%2Faxolotl) Appearance settings \n- Platform \n    - AI CODE CREATION \n        -  [GitHub Copilot Write better code with AI ](https://github.com/features/copilot) \n        -  [GitHub Spark Build and deploy intelligent apps ](https://github.com/features/spark) \n        -  [GitHub Models Manage and compare prompts ](https://github.com/features/models) \n        -  [MCP Registry New Integrate external tools ](https://github.com/mcp) \n\n    - DEVELOPER WORKFLOWS \n        -  [Actions Automate any workflow ](https://github.com/features/actions) \n        -  [Codespaces Instant dev environments ](https://github.com/features/codespaces) \n        -  [Issues Plan and track work ](https://github.com/features/issues) \n        -  [Code Review Manage code changes ](https://github.com/features/code-review) \n\n    - APPLICATION SECURITY \n        -  [GitHub Advanced Security Find and fix vulnerabilities ](https://github.com/security/advanced-security) \n        -  [Code security Secure your code as you build ](https://github.com/security/advanced-security/code-security) \n        -  [Secret protection Stop leaks before they start ](https://github.com/security/advanced-security/secret-protection) \n\n    - EXPLORE \n        -  [Why GitHub ](https://github.com/why-github) \n        -  [Documentation ](https://docs.github.com) \n        -  [Blog ](https://github.blog) \n        -  [Changelog ](https://github.blog/changelog) \n        -  [Marketplace ](https://github.com/marketplace) \n\n [View all features ](https://github.com/features) \n- Solutions \n    - BY COMPANY SIZE \n        -  [Enterprises ](https://github.com/enterprise) \n        -  [Small and medium teams ](https://github.com/team) \n        -  [Startups ](https://github.com/enterprise/startups) \n        -  [Nonprofits ](https://github.com/solutions/industry/nonprofits) \n\n    - BY USE CASE \n        -  [App Modernization ](https://github.com/solutions/use-case/app-modernization) \n        -  [DevSecOps ](https://github.com/solutions/use-case/devsecops) \n        -  [DevOps ](https://github.com/solutions/use-case/devops) \n        -  [CI/CD ](https://github.com/solutions/use-case/ci-cd) \n        -  [View all use cases ](https://github.com/solutions/use-case) \n\n    - BY INDUSTRY \n        -  [Healthcare ](https://github.com/solutions/industry/healthcare) \n        -  [Financial services ](https://github.com/solutions/industry/financial-services) \n        -  [Manufacturing ](https://github.com/solutions/industry/manufacturing) \n        -  [Government ](https://github.com/solutions/industry/government) \n        -  [View all industries ](https://github.com/solutions/industry) \n\n [View all solutions ](https://github.com/solutions) \n- Resources \n    - EXPLORE BY TOPIC \n        -  [AI ](https://github.com/resources/articles?topic=ai) \n        -  [Software Development ](https://github.com/resources/articles?topic=software-development) \n        -  [DevOps ](https://github.com/resources/articles?topic=devops) \n        -  [Security ](https://github.com/resources/articles?topic=security) \n        -  [View all topics ](https://github.com/resources/articles) \n\n    - EXPLORE BY TYPE \n        -  [Customer stories ](https://github.com/customer-stories) \n        -  [Events & webinars ](https://github.com/resources/events) \n        -  [Ebooks & reports ](https://github.com/resources/whitepapers) \n        -  [Business insights ](https://github.com/solutions/executive-insights) \n        -  [GitHub Skills ](https://skills.github.com) \n\n    - SUPPORT & SERVICES \n        -  [Documentation ](https://docs.github.com) \n        -  [Customer support ](https://support.github.com) \n        -  [Community forum ](https://github.com/orgs/community/discussions) \n        -  [Trust center ](https://github.com/trust-center) \n        -  [Partners ](https://github.com/partners) \n\n [View all resources ](https://github.com/resources) \n- Open Source \n    - COMMUNITY \n        -  [GitHub Sponsors Fund open source developers ](https://github.com/sponsors) \n\n    - PROGRAMS \n        -  [Security Lab ](https://securitylab.github.com) \n        -  [Maintainer Community ](https://maintainers.github.com) \n        -  [Accelerator ](https://github.com/accelerator) \n        -  [GitHub Stars ](https://stars.github.com) \n        -  [Archive Program ](https://archiveprogram.github.com) \n\n    - REPOSITORIES \n        -  [Topics ](https://github.com/topics) \n        -  [Trending ](https://github.com/trending) \n        -  [Collections ](https://github.com/collections) \n\n\n- Enterprise \n    - ENTERPRISE SOLUTIONS \n        -  [Enterprise platform AI-powered developer platform ](https://github.com/enterprise) \n\n    - AVAILABLE ADD-ONS \n        -  [GitHub Advanced Security Enterprise-grade security features ](https://github.com/security/advanced-security) \n        -  [Copilot for Business Enterprise-grade AI features ](https://github.com/features/copilot/copilot-business) \n        -  [Premium Support Enterprise-grade 24/7 support ](https://github.com/premium-support) \n\n\n-  [Pricing ](https://github.com/pricing) \nSearch or jump to... \n\n# Search code, repositories, users, issues, pull requests...  Search  Clear \n\n [Search syntax tips ](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax) \n\n#  Provide feedback  \n\nWe read every piece of feedback, and take your input very seriously. Include my email address so I can be contacted  Cancel   Submit feedback  \n\n#  Saved searches  \n\n## Use saved searches to filter your results more quickly Name Query \n\n To see all available qualifiers, see our   [documentation ](https://docs.github.com/search-github/github-code-search/understanding-github-code-search-syntax) .   Cancel   Create saved search   [ Sign in  ](https://github.com/login?return_to=https%3A%2F%2Fgithub.com%2Faxolotl-ai-cloud%2Faxolotl)  [ Sign up  ](https://github.com/signup?ref_cta=Sign+up&ref_loc=header+logged+out&ref_page=%2F%3Cuser-name%3E%2F%3Crepo-name%3E&source=header-repo&source_repo=axolotl-ai-cloud%2Faxolotl) Appearance settings Resetting focus You signed in with another tab or window.   [Reload ]()  to refresh your session. You signed out in another tab or window.   [Reload ]()  to refresh your session. You switched accounts on another tab or window.   [Reload ]()  to refresh your session. Dismiss alert {{ message }}  [ axolotl-ai-cloud  ](https://github.com/axolotl-ai-cloud) /  ** [axolotl ](https://github.com/axolotl-ai-cloud/axolotl) ** Public \n-  [ Sponsor  ](https://github.com/sponsors/axolotl-ai-cloud) \n-  [Notifications  ](https://github.com/login?return_to=%2Faxolotl-ai-cloud%2Faxolotl) You must be signed in to change notification settings \n-  [Fork  1.3k ](https://github.com/login?return_to=%2Faxolotl-ai-cloud%2Faxolotl) \n-  [ Star  11.9k ](https://github.com/login?return_to=%2Faxolotl-ai-cloud%2Faxolotl) \n\n-  [Code ](https://github.com/axolotl-ai-cloud/axolotl) \n-  [Issues 120 ](https://github.com/axolotl-ai-cloud/axolotl/issues) \n-  [Pull requests 97 ](https://github.com/axolotl-ai-cloud/axolotl/pulls) \n-  [Discussions ](https://github.com/axolotl-ai-cloud/axolotl/discussions) \n-  [Actions ](https://github.com/axolotl-ai-cloud/axolotl/actions) \n-  [Projects ](https://github.com/axolotl-ai-cloud/axolotl/projects) \n-  [Security and quality 0 ](https://github.com/axolotl-ai-cloud/axolotl/security) \n-  [Insights ](https://github.com/axolotl-ai-cloud/axolotl/pulse) \nAdditional navigation options \n-  [ Code  ](https://github.com/axolotl-ai-cloud/axolotl) \n-  [ Issues  ](https://github.com/axolotl-ai-cloud/axolotl/issues) \n-  [ Pull requests  ](https://github.com/axolotl-ai-cloud/axolotl/pulls) \n-  [ Discussions  ](https://github.com/axolotl-ai-cloud/axolotl/discussions) \n-  [ Actions  ](https://github.com/axolotl-ai-cloud/axolotl/actions) \n-  [ Projects  ](https://github.com/axolotl-ai-cloud/axolotl/projects) \n-  [ Security and quality  ](https://github.com/axolotl-ai-cloud/axolotl/security) \n-  [ Insights  ](https://github.com/axolotl-ai-cloud/axolotl/pulse) \n [](https://github.com/axolotl-ai-cloud/axolotl) \n\n# axolotl-ai-cloud/axolotl main  [ **313  ** Branches ](https://github.com/axolotl-ai-cloud/axolotl/branches)  [ **32  ** Tags ](https://github.com/axolotl-ai-cloud/axolotl/tags)  [](https://github.com/axolotl-ai-cloud/axolotl/branches)  [](https://github.com/axolotl-ai-cloud/axolotl/tags) Go to file Code Open more actions menu \n\n## Folders and files \n\n\n| Name  | Name  | Last commit message  | Last commit date  |\n | ---- | ---- | ---- | ---- |\n| \n\n## Latest commit ![winglian](https://avatars.githubusercontent.com/u/381258?v=4&size=40)![NanoCode012](https://avatars.githubusercontent.com/u/9899957?v=4&size=40) [winglian ](https://github.com/axolotl-ai-cloud/axolotl/commits?author=winglian) and  [NanoCode012 ](https://github.com/axolotl-ai-cloud/axolotl/commits?author=NanoCode012)  [feat: add pytorch 2.11 to base images ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/d7cb1c9d03dace6e3db34ed5b8ee908eae671a7a)  [#3621 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3621)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/d7cb1c9d03dace6e3db34ed5b8ee908eae671a7a) Open commit details failure May 13, 20262 days ago  [d7cb1c9 ](https://github.com/axolotl-ai-cloud/axolotl/commit/d7cb1c9d03dace6e3db34ed5b8ee908eae671a7a)  ·  May 13, 20262 days ago \n\n## History  [2,744 Commits ](https://github.com/axolotl-ai-cloud/axolotl/commits/main/) Open commit details  [](https://github.com/axolotl-ai-cloud/axolotl/commits/main/) 2,744 Commits  |\n|  [.github ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/.github)  |  [.github ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/.github)  |  [feat: add pytorch 2.11 to base images ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/d7cb1c9d03dace6e3db34ed5b8ee908eae671a7a)  [#3621 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3621)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/d7cb1c9d03dace6e3db34ed5b8ee908eae671a7a)  | May 13, 20262 days ago  |\n|  [.runpod ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/.runpod)  |  [.runpod ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/.runpod)  |  [Changes from dataset_processes to dataset_num_proc ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/530a0c0bf0759d59e87684349f11f042657397fa)  [#3352 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3352)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/530a0c0bf0759d59e87684349f11f042657397fa)  | Feb 10, 20263 months ago  |\n|  [.vscode ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/.vscode)  |  [.vscode ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/.vscode)  |  [feat: enable trl's autounwrap ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/b432889256bf6c127b0dcc45c6c32fb0834ddecc)  [#1060 ](https://github.com/axolotl-ai-cloud/axolotl/pull/1060)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/b432889256bf6c127b0dcc45c6c32fb0834ddecc)  | Jan 11, 20243 years ago  |\n|  [benchmarks ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/benchmarks)  |  [benchmarks ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/benchmarks)  |  [Scattermoe LoRA optimizations ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/1fc86d5295eb4e10b9af2550e0b50b1c1ba540f4)  [#3513 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3513)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/1fc86d5295eb4e10b9af2550e0b50b1c1ba540f4)  | Mar 20, 20262 months ago  |\n|  [cicd ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/cicd)  |  [cicd ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/cicd)  |  [feat: move to uv first ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/9de5b76336c5b658614e4d0c03cc45b303d7d8d4)  [#3545 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3545)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/9de5b76336c5b658614e4d0c03cc45b303d7d8d4)  | Apr 21, 20263 weeks ago  |\n|  [deepspeed_configs ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/deepspeed_configs)  |  [deepspeed_configs ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/deepspeed_configs)  |  [revert renaming of deepspeed stage3 args that use auto ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/631268a0caa18250a30a1c837c4d7f4d9505adc0)  [#2964 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2964)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/631268a0caa18250a30a1c837c4d7f4d9505adc0)  | Jul 22, 202510 months ago  |\n|  [devtools ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/devtools)  |  [devtools ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/devtools)  |  [feat:add support dataset_num_processes ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/cd856b45b168b41f2286f59efe13facc55a36eb5)  [#3129 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3129)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/cd856b45b168b41f2286f59efe13facc55a36eb5)  | Oct 13, 20257 months ago  |\n|  [docker ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/docker)  |  [docker ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/docker)  |  [Fix cu130 LD_LIBRARY_PATH startup ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/52c78432f477212a465c717dea28f2a5511dd40f)  [#3648 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3648)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/52c78432f477212a465c717dea28f2a5511dd40f)  | May 13, 20262 days ago  |\n|  [docs ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/docs)  |  [docs ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/docs)  |  [feat: support field_messages in multimodal collator ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/4f855f6282b7c9ddd3e33dfe8cb332108b1ab617)  [#3628 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3628)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/4f855f6282b7c9ddd3e33dfe8cb332108b1ab617)  | May 13, 20262 days ago  |\n|  [examples ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples)  |  [examples ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples)  |  [qwen fft mutli ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/73d782b1eaeac0b7f4792e8414810925e0f28651)  [#3605 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3605)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/73d782b1eaeac0b7f4792e8414810925e0f28651)  | May 13, 20262 days ago  |\n|  [image ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/image)  |  [image ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/image)  |  [Readme updates v2 ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/c07bd2fa65750573e2b50211081368b3ea3d497f)  [#2078 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2078)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/c07bd2fa65750573e2b50211081368b3ea3d497f)  | Nov 18, 20242 years ago  |\n|  [scripts ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/scripts)  |  [scripts ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/scripts)  |  [Fix cu130 LD_LIBRARY_PATH startup ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/52c78432f477212a465c717dea28f2a5511dd40f)  [#3648 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3648)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/52c78432f477212a465c717dea28f2a5511dd40f)  | May 13, 20262 days ago  |\n|  [src/ axolotl ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/src/axolotl)  |  [src/ axolotl ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/src/axolotl)  |  [feat: support field_messages in multimodal collator ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/4f855f6282b7c9ddd3e33dfe8cb332108b1ab617)  [#3628 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3628)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/4f855f6282b7c9ddd3e33dfe8cb332108b1ab617)  | May 13, 20262 days ago  |\n|  [tests ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/tests)  |  [tests ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/tests)  |  [feat: support field_messages in multimodal collator ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/4f855f6282b7c9ddd3e33dfe8cb332108b1ab617)  [#3628 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3628)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/4f855f6282b7c9ddd3e33dfe8cb332108b1ab617)  | May 13, 20262 days ago  |\n|  [.axolotl-complete.bash ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.axolotl-complete.bash)  |  [.axolotl-complete.bash ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.axolotl-complete.bash)  |  [Autocomplete axolotl CLI ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/3a208cfd84a826eedc7eb6ec6c67546a2c28b8b0)  [#2955 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2955)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/3a208cfd84a826eedc7eb6ec6c67546a2c28b8b0)  | Jul 22, 202510 months ago  |\n|  [.bandit ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.bandit)  |  [.bandit ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.bandit)  |  [Add ruff, remove black, isort, flake8, pylint ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/79ddaebe9a6af7efefebdbb54772d11d09561786)  [#3092 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3092)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/79ddaebe9a6af7efefebdbb54772d11d09561786)  | Aug 24, 20259 months ago  |\n|  [.coderabbit.yaml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.coderabbit.yaml)  |  [.coderabbit.yaml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.coderabbit.yaml)  |  [Update .coderabbit.yaml ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/dc338c3b0eccdfd12b2fdd83eeff1e48c7965bc1)  [#3109 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3109)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/dc338c3b0eccdfd12b2fdd83eeff1e48c7965bc1)  | Aug 27, 20259 months ago  |\n|  [.coveragerc ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.coveragerc)  |  [.coveragerc ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.coveragerc)  |  [adding codecov reporting ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/f776f889a164b4440debd3c7d2f35f8356aaea44)  [#2372 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2372)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/f776f889a164b4440debd3c7d2f35f8356aaea44)  | Apr 16, 2025last year  |\n|  [.editorconfig ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.editorconfig)  |  [.editorconfig ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.editorconfig)  |  [WIP for axolotl trainer ](https://github.com/axolotl-ai-cloud/axolotl/commit/ce24f5e246e690bebbb06a6cf2793ce6132f219f)  | Apr 14, 20233 years ago  |\n|  [.gitattributes ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.gitattributes)  |  [.gitattributes ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.gitattributes)  |  [make it work with pythia in the cloud ](https://github.com/axolotl-ai-cloud/axolotl/commit/8d959a7e2639f3ddfc6f1fa1183d7a16acf2764a)  | Apr 14, 20233 years ago  |\n|  [.gitignore ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.gitignore)  |  [.gitignore ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.gitignore)  |  [Debug log, logging improvements ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/4065bc14c616e12c4da037c01de8a0defd9e7c10)  [#3159 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3159)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/4065bc14c616e12c4da037c01de8a0defd9e7c10)  | Sep 17, 20258 months ago  |\n|  [.mypy.ini ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.mypy.ini)  |  [.mypy.ini ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.mypy.ini)  |  [Liger Kernel integration ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/1f686c576c40a6dd7c8c785a133ac8fc09174b2e)  [#1861 ](https://github.com/axolotl-ai-cloud/axolotl/pull/1861)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/1f686c576c40a6dd7c8c785a133ac8fc09174b2e)  | Aug 23, 20242 years ago  |\n|  [.pre-commit-config.yaml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.pre-commit-config.yaml)  |  [.pre-commit-config.yaml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.pre-commit-config.yaml)  |  [chore: update pre-commit hooks ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/438ea7b045bda37c49a257d3e07447efd0b4a885)  [#3567 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3567)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/438ea7b045bda37c49a257d3e07447efd0b4a885)  | Apr 1, 2026last month  |\n|  [AGENTS.md ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/AGENTS.md)  |  [AGENTS.md ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/AGENTS.md)  |  [DPO support loss types ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/90090fa9e88a21054f05998ab1143cbce5a2ebd4)  [#3566 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3566)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/90090fa9e88a21054f05998ab1143cbce5a2ebd4)  | Apr 23, 20263 weeks ago  |\n|  [CITATION.cff ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/CITATION.cff)  |  [CITATION.cff ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/CITATION.cff)  |  [SEO go brrr ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/0401a15888e480d03c7cd0fb6439b27e0dacd3a0)  [#3153 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3153)  [) [skip-ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/0401a15888e480d03c7cd0fb6439b27e0dacd3a0)  | Sep 12, 20258 months ago  |\n|  [CNAME ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/CNAME)  |  [CNAME ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/CNAME)  |  [feat: add CNAME ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/16823e1de678098a5d084e52f70e3ddee1051412)  [#2513 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2513)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/16823e1de678098a5d084e52f70e3ddee1051412)  | Apr 10, 2025last year  |\n|  [FAQS.md ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/FAQS.md)  |  [FAQS.md ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/FAQS.md)  |  [Update FAQS.md ](https://github.com/axolotl-ai-cloud/axolotl/commit/e3e7b52a5b0549fc6abb75ab6d3aa377d9478924)  | Jun 11, 20233 years ago  |\n|  [LICENSE ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/LICENSE)  |  [LICENSE ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/LICENSE)  |  [add apache 2.0 license ](https://github.com/axolotl-ai-cloud/axolotl/commit/5cce2a42ffc408e8c1018583857f1eb91a992107)  | Jul 21, 20233 years ago  |\n|  [MANIFEST.in ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/MANIFEST.in)  |  [MANIFEST.in ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/MANIFEST.in)  |  [feat: move to uv first ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/9de5b76336c5b658614e4d0c03cc45b303d7d8d4)  [#3545 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3545)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/9de5b76336c5b658614e4d0c03cc45b303d7d8d4)  | Apr 21, 20263 weeks ago  |\n|  [README.md ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/README.md)  |  [README.md ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/README.md)  |  [feat: Add Mistral Medium 3.5 ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/ebbd7fa8473c2412cfd78b4f6cc0233e5d907970)  [#3633 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3633)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/ebbd7fa8473c2412cfd78b4f6cc0233e5d907970)  | Apr 29, 20262 weeks ago  |\n|  [VERSION ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/VERSION)  |  [VERSION ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/VERSION)  |  [fix: docker build failing ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/17fc747f99e24eeb7cb295333bf7026e0d5f2bdd)  [#3622 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3622)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/17fc747f99e24eeb7cb295333bf7026e0d5f2bdd)  | Apr 24, 20263 weeks ago  |\n|  [_quarto.yml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/_quarto.yml)  |  [_quarto.yml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/_quarto.yml)  |  [Feat: Add bitnet integration ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/e662972a2912b395da4817bf13d73b7d4a6d9137)  [#3634 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3634)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/e662972a2912b395da4817bf13d73b7d4a6d9137)  | Apr 30, 20262 weeks ago  |\n|  [codecov.yml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/codecov.yml)  |  [codecov.yml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/codecov.yml)  |  [increase rtol, codecov informational only, don't silently fail errors\u2026 ](https://github.com/axolotl-ai-cloud/axolotl/commit/0e583efeaa29dc584344d03820afda6612fb6fd2)  | Mar 22, 20262 months ago  |\n|  [docker-compose.yaml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/docker-compose.yaml)  |  [docker-compose.yaml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/docker-compose.yaml)  |  [add git environment variables to compose: avoid checkout failure erro\u2026 ](https://github.com/axolotl-ai-cloud/axolotl/commit/78ee2cdab28f0d346b7039308f35bf9cc512bc94)  | Sep 8, 20233 years ago  |\n|  [favicon.jpg ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/favicon.jpg)  |  [favicon.jpg ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/favicon.jpg)  |  [update favicon ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/da8f6c32b96aea79c36548b5d51790defffecfab)  [#2801 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2801)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/da8f6c32b96aea79c36548b5d51790defffecfab)  | Jun 17, 202511 months ago  |\n|  [index.qmd ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/index.qmd)  |  [index.qmd ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/index.qmd)  |  [Revert test update to index.qmd ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/32a78902315ddeef6b32d0f18a4a37dfa7333038)  [#2995 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2995)  [) [skip ci] ](https://github.com/axolotl-ai-cloud/axolotl/commit/32a78902315ddeef6b32d0f18a4a37dfa7333038)  | Jul 31, 202510 months ago  |\n|  [pyproject.toml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/pyproject.toml)  |  [pyproject.toml ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/pyproject.toml)  |  [fix: docker build failing ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/17fc747f99e24eeb7cb295333bf7026e0d5f2bdd)  [#3622 ](https://github.com/axolotl-ai-cloud/axolotl/pull/3622)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/17fc747f99e24eeb7cb295333bf7026e0d5f2bdd)  | Apr 24, 20263 weeks ago  |\n|  [styles.css ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/styles.css)  |  [styles.css ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/styles.css)  |  [Autodoc generation with quartodoc ( ](https://github.com/axolotl-ai-cloud/axolotl/commit/113e9cd19323d66f38013616be54cdb4b3a22393)  [#2419 ](https://github.com/axolotl-ai-cloud/axolotl/pull/2419)  [) ](https://github.com/axolotl-ai-cloud/axolotl/commit/113e9cd19323d66f38013616be54cdb4b3a22393)  | Mar 21, 2025last year  |\n| View all files  |\n\n## Repository files navigation \n-  [README ](https://github.com/axolotl-ai-cloud/axolotl/#) \n-  [Code of conduct ](https://github.com/axolotl-ai-cloud/axolotl/#) \n-  [Contributing ](https://github.com/axolotl-ai-cloud/axolotl/#) \n-  [Apache-2.0 license ](https://github.com/axolotl-ai-cloud/axolotl/#) \n-  [Security ](https://github.com/axolotl-ai-cloud/axolotl/#) \n\n\n![Axolotl](https://raw.githubusercontent.com/axolotl-ai-cloud/axolotl/887513285d98132142bf5db2a74eb5e0928787f1/image/axolotl_logo_digital_black.svg)\n\n **A Free and Open Source LLM Fine-tuning Framework ** \n\n [![GitHub License](https://camo.githubusercontent.com/7da0c70d14204a28c582f3fce261e0ef0519de8dbe52f681a7e262c92c2bf1f4/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f61786f6c6f746c2d61692d636c6f75642f61786f6c6f746c2e7376673f636f6c6f723d626c7565)]()  [![tests](https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/tests.yml/badge.svg)](https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/tests.yml/badge.svg)  [![codecov](https://camo.githubusercontent.com/ee65975667c51a59343199454ba1634c18a900b0a9faad55675167cb674e24e3/68747470733a2f2f636f6465636f762e696f2f67682f61786f6c6f746c2d61692d636c6f75642f61786f6c6f746c2f6272616e63682f6d61696e2f67726170682f62616467652e737667)](https://codecov.io/gh/axolotl-ai-cloud/axolotl)  [![Releases](https://camo.githubusercontent.com/32f4af887842c19e398bbab79d877a9a3d6271f384489dd25455a5e2494f567a/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f72656c656173652f61786f6c6f746c2d61692d636c6f75642f61786f6c6f746c2e737667)](https://github.com/axolotl-ai-cloud/axolotl/releases)  [](https://github.com/axolotl-ai-cloud/axolotl/graphs/contributors)  [![GitHub Repo stars](https://camo.githubusercontent.com/eafbec52266784701d6ebb0043b63dc5371f9b46cceba918f311e2c2b6ee9c5e/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f73746172732f61786f6c6f746c2d61692d636c6f75642f61786f6c6f746c)]()  [![discord](https://camo.githubusercontent.com/37727ecbba4188a458fc7012520215f99a7e9509362a15720ca8ca0cb59769ee/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f646973636f72642d3732383964612e7376673f7374796c653d666c61742d737175617265266c6f676f3d646973636f7264)](https://discord.com/invite/HhrNrHJPRb)  [![twitter](https://camo.githubusercontent.com/12c6ca5de6f8a018b2caf6e464811239ee07bcce9856650b33b1ac133ba5f90e/68747470733a2f2f696d672e736869656c64732e696f2f747769747465722f666f6c6c6f772f61786f6c6f746c5f61693f7374796c653d736f6369616c)](https://twitter.com/axolotl_ai)  [![google-colab](https://camo.githubusercontent.com/eff96fda6b2e0fff8cdf2978f89d61aa434bb98c00453ae23dd0aab8d1451633/68747470733a2f2f636f6c61622e72657365617263682e676f6f676c652e636f6d2f6173736574732f636f6c61622d62616467652e737667)](https://colab.research.google.com/github/axolotl-ai-cloud/axolotl/blob/main/examples/colab-notebooks/colab-axolotl-example.ipynb)  [![tests-nightly](https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/tests-nightly.yml/badge.svg)](https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/tests-nightly.yml/badge.svg)  [![multigpu-semi-weekly tests](https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/multi-gpu-e2e.yml/badge.svg)](https://github.com/axolotl-ai-cloud/axolotl/actions/workflows/multi-gpu-e2e.yml/badge.svg) \n\n## 🎉 Latest Updates  [](https://github.com/axolotl-ai-cloud/axolotl/#-latest-updates) \n- 2026/04:  \n    - New model support has been added in Axolotl for   [Mistral Medium 3.5 ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/mistral-medium-3_5)  and   [Gemma 4 ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/gemma4) . \n    - Axolotl is now   [uv-first ](https://github.com/axolotl-ai-cloud/axolotl/pull/3545)  and has   [SonicMoE fused LoRA ](https://github.com/axolotl-ai-cloud/axolotl/pull/3519)  support. \n\n- 2026/03:  \n    - New model support has been added in Axolotl for   [Mistral Small 4 ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/mistral4) ,   [Qwen3.5, Qwen3.5 MoE ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/qwen3.5) ,   [GLM-4.7-Flash ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/glm47-flash) ,   [GLM-4.6V ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/glm46v) , and   [GLM-4.5-Air ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/glm45) . \n    -  [MoE expert quantization ](https://docs.axolotl.ai/docs/expert_quantization.html)  support (via   `quantize_moe_experts: true `) greatly reduces VRAM when training MoE models (FSDP2 compat). \n\n- 2026/02:  \n    -  [ScatterMoE LoRA ](https://github.com/axolotl-ai-cloud/axolotl/pull/3410)  support. LoRA fine-tuning directly on MoE expert weights using custom Triton kernels. \n    - Axolotl now has support for   [SageAttention ](https://github.com/axolotl-ai-cloud/axolotl/pull/2823)  and   [GDPO ](https://github.com/axolotl-ai-cloud/axolotl/pull/3353)  (Generalized DPO). \n\n- 2026/01:  \n    - New integration for   [EAFT ](https://github.com/axolotl-ai-cloud/axolotl/pull/3366)  (Entropy-Aware Focal Training), weights loss by entropy of the top-k logit distribution, and   [Scalable Softmax ](https://github.com/axolotl-ai-cloud/axolotl/pull/3338) , improves long context in attention. \n\n- 2025/12:  \n    - Axolotl now includes support for   [Kimi-Linear ](https://docs.axolotl.ai/docs/models/kimi-linear.html) ,   [Plano-Orchestrator ](https://docs.axolotl.ai/docs/models/plano.html) ,   [MiMo ](https://docs.axolotl.ai/docs/models/mimo.html) ,   [InternVL 3.5 ](https://docs.axolotl.ai/docs/models/internvl3_5.html) ,   [Olmo3 ](https://docs.axolotl.ai/docs/models/olmo3.html) ,   [Trinity ](https://docs.axolotl.ai/docs/models/trinity.html) , and   [Ministral3 ](https://docs.axolotl.ai/docs/models/ministral3.html) . \n    -  [Distributed Muon Optimizer ](https://github.com/axolotl-ai-cloud/axolotl/pull/3264)  support has been added for FSDP2 pretraining. \n\n- 2025/10: New model support has been added in Axolotl for:   [Qwen3 Next ](https://docs.axolotl.ai/docs/models/qwen3-next.html) ,   [Qwen2.5-vl, Qwen3-vl ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/qwen2_5-vl) ,   [Qwen3, Qwen3MoE ](https://docs.axolotl.ai/docs/models/qwen3.html) ,   [Granite 4 ](https://docs.axolotl.ai/docs/models/granite4.html) ,   [HunYuan ](https://docs.axolotl.ai/docs/models/hunyuan.html) ,   [Magistral 2509 ](https://docs.axolotl.ai/docs/models/magistral/vision.html) ,   [Apertus ](https://docs.axolotl.ai/docs/models/apertus.html) , and   [Seed-OSS ](https://docs.axolotl.ai/docs/models/seed-oss.html) . \nExpand older updates \n- 2025/09: Axolotl now has text diffusion training. Read more   [here ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/src/axolotl/integrations/diffusion) . \n- 2025/08: QAT has been updated to include NVFP4 support. See   [PR ](https://github.com/axolotl-ai-cloud/axolotl/pull/3107) . \n- 2025/07:  \n    - ND Parallelism support has been added into Axolotl. Compose Context Parallelism (CP), Tensor Parallelism (TP), and Fully Sharded Data Parallelism (FSDP) within a single node and across multiple nodes. Check out the   [blog post ](https://huggingface.co/blog/accelerate-nd-parallel)  for more info. \n    - Axolotl adds more models:   [GPT-OSS ](https://docs.axolotl.ai/docs/models/gpt-oss.html) ,   [Gemma 3n ](https://docs.axolotl.ai/docs/models/gemma3n.html) ,   [Liquid Foundation Model 2 (LFM2) ](https://docs.axolotl.ai/docs/models/LiquidAI.html) , and   [Arcee Foundation Models (AFM) ](https://docs.axolotl.ai/docs/models/arcee.html) . \n    - FP8 finetuning with fp8 gather op is now possible in Axolotl via   `torchao `. Get started   [here ](https://docs.axolotl.ai/docs/mixed_precision.html#sec-fp8) ! \n    -  [Voxtral ](https://docs.axolotl.ai/docs/models/voxtral.html) ,   [Magistral 1.1 ](https://docs.axolotl.ai/docs/models/magistral.html) , and   [Devstral ](https://docs.axolotl.ai/docs/models/devstral.html)  with mistral-common tokenizer support has been integrated in Axolotl! \n    - TiledMLP support for single-GPU to multi-GPU training with DDP, DeepSpeed and FSDP support has been added to support Arctic Long Sequence Training. (ALST). See   [examples ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/alst)  for using ALST with Axolotl! \n\n- 2025/06: Magistral with mistral-common tokenizer support has been added to Axolotl. See   [docs ](https://docs.axolotl.ai/docs/models/magistral.html)  to start training your own Magistral models with Axolotl! \n- 2025/05: Quantization Aware Training (QAT) support has been added to Axolotl. Explore the   [docs ](https://docs.axolotl.ai/docs/qat.html)  to learn more! \n- 2025/04: Llama 4 support has been added in Axolotl. See   [docs ](https://docs.axolotl.ai/docs/models/llama-4.html)  to start training your own Llama 4 models with Axolotl's linearized version! \n- 2025/03: Axolotl has implemented Sequence Parallelism (SP) support. Read the   [blog ](https://huggingface.co/blog/axolotl-ai-co/long-context-with-sequence-parallelism-in-axolotl)  and   [docs ](https://docs.axolotl.ai/docs/sequence_parallelism.html)  to learn how to scale your context length when fine-tuning. \n- 2025/03: (Beta) Fine-tuning Multimodal models is now supported in Axolotl. Check out the   [docs ](https://docs.axolotl.ai/docs/multimodal.html)  to fine-tune your own! \n- 2025/02: Axolotl has added LoRA optimizations to reduce memory usage and improve training speed for LoRA and QLoRA in single GPU and multi-GPU training (DDP and DeepSpeed). Jump into the   [docs ](https://docs.axolotl.ai/docs/lora_optims.html)  to give it a try. \n- 2025/02: Axolotl has added GRPO support. Dive into our   [blog ](https://huggingface.co/blog/axolotl-ai-co/training-llms-w-interpreter-feedback-wasm)  and   [GRPO example ](https://github.com/axolotl-ai-cloud/grpo_code)  and have some fun! \n- 2025/01: Axolotl has added Reward Modelling / Process Reward Modelling fine-tuning support. See   [docs ](https://docs.axolotl.ai/docs/reward_modelling.html) . \n\n\n## ✨ Overview  [](https://github.com/axolotl-ai-cloud/axolotl/#-overview) \n\nAxolotl is a free and open-source tool designed to streamline post-training and fine-tuning for the latest large language models (LLMs). \n\nFeatures: \n-  **Multiple Model Support ** : Train various models like GPT-OSS, LLaMA, Mistral, Mixtral, Pythia, and many more models available on the Hugging Face Hub. \n-  **Multimodal Training ** : Fine-tune vision-language models (VLMs) including LLaMA-Vision, Qwen2-VL, Pixtral, LLaVA, SmolVLM2, GLM-4.6V, InternVL 3.5, Gemma 3n, and audio models like Voxtral with image, video, and audio support. \n-  **Training Methods ** : Full fine-tuning, LoRA, QLoRA, GPTQ, QAT, Preference Tuning (DPO, IPO, KTO, ORPO), RL (GRPO, GDPO), and Reward Modelling (RM) / Process Reward Modelling (PRM). \n-  **Easy Configuration ** : Re-use a single YAML configuration file across the full fine-tuning pipeline: dataset preprocessing, training, evaluation, quantization, and inference. \n-  **Performance Optimizations ** :   [Multipacking ](https://docs.axolotl.ai/docs/multipack.html) ,   [Flash Attention 2/3/4 ](https://docs.axolotl.ai/docs/attention.html#flash-attention) ,   [Xformers ](https://docs.axolotl.ai/docs/attention.html#xformers) ,   [Flex Attention ](https://docs.axolotl.ai/docs/attention.html#flex-attention) ,   [SageAttention ](https://docs.axolotl.ai/docs/attention.html#sageattention) ,   [Liger Kernel ](https://docs.axolotl.ai/docs/custom_integrations.html#liger-kernels) ,   [Cut Cross Entropy ](https://docs.axolotl.ai/docs/custom_integrations.html#cut-cross-entropy) ,   [ScatterMoE ](https://docs.axolotl.ai/docs/custom_integrations.html#kernels-integration) ,   [Sequence Parallelism (SP) ](https://docs.axolotl.ai/docs/sequence_parallelism.html) ,   [LoRA optimizations ](https://docs.axolotl.ai/docs/lora_optims.html) ,   [Multi-GPU training (FSDP1, FSDP2, DeepSpeed) ](https://docs.axolotl.ai/docs/multi-gpu.html) ,   [Multi-node training (Torchrun, Ray) ](https://docs.axolotl.ai/docs/multi-node.html) , and many more! \n-  **Flexible Dataset Handling ** : Load from local, HuggingFace, and cloud (S3, Azure, GCP, OCI) datasets. \n-  **Cloud Ready ** : We ship   [Docker images ](https://hub.docker.com/u/axolotlai)  and also   [PyPI packages ](https://pypi.org/project/axolotl/)  for use on cloud platforms and local hardware. \n\n\n## 🚀 Quick Start - LLM Fine-tuning in Minutes  [](https://github.com/axolotl-ai-cloud/axolotl/#-quick-start---llm-fine-tuning-in-minutes) \n\n **Requirements ** : \n- NVIDIA GPU (Ampere or newer for   `bf16 ` and Flash Attention) or AMD GPU \n- Python >=3.11 (3.12 recommended) \n- PyTorch ≥2.9.1 \n\n\n### Google Colab  [](https://github.com/axolotl-ai-cloud/axolotl/#google-colab) \n\n [![Open In Colab](https://camo.githubusercontent.com/eff96fda6b2e0fff8cdf2978f89d61aa434bb98c00453ae23dd0aab8d1451633/68747470733a2f2f636f6c61622e72657365617263682e676f6f676c652e636f6d2f6173736574732f636f6c61622d62616467652e737667)](https://colab.research.google.com/github/axolotl-ai-cloud/axolotl/blob/main/examples/colab-notebooks/colab-axolotl-example.ipynb#scrollTo=msOCO4NRmRLa) \n\n### Installation  [](https://github.com/axolotl-ai-cloud/axolotl/#installation) #  install uv if you don't already have it installed (restart shell after)  curl -LsSf https://astral.sh/uv/install.sh  |  sh  #  change depending on system export  UV_TORCH_BACKEND=cu128  #  create a new virtual environment  uv venv --python 3.12  source  .venv/bin/activate uv pip install torch==2.10.0 torchvision uv pip install --no-build-isolation axolotl[deepspeed]  #  Download example axolotl configs, deepspeed configs  axolotl fetch examples axolotl fetch deepspeed_configs  #  OPTIONAL \n\n#### Using Docker  [](https://github.com/axolotl-ai-cloud/axolotl/#using-docker) \n\nInstalling with Docker can be less error prone than installing in your own environment. docker run --gpus  ' \"all\" '  --ipc=host --rm -it axolotlai/axolotl:main-latest \n\nOther installation approaches are described   [here ](https://docs.axolotl.ai/docs/installation.html) . \n\n#### Cloud Providers  [](https://github.com/axolotl-ai-cloud/axolotl/#cloud-providers) Details \n-  [RunPod ](https://runpod.io/gsc?template=v2ickqhz9s&ref=6i7fkpdz) \n-  [Vast.ai ](https://cloud.vast.ai?ref_id=62897&template_id=bdd4a49fa8bce926defc99471864cace&utm_source=github&utm_medium=developer_community&utm_campaign=template_launch_axolotl&utm_content=readme) \n-  [PRIME Intellect ](https://app.primeintellect.ai/dashboard/create-cluster?image=axolotl&location=Cheapest&security=Cheapest&show_spot=true) \n-  [Modal ](https://www.modal.com?utm_source=github&utm_medium=github&utm_campaign=axolotl) \n-  [Novita ](https://novita.ai/gpus-console?templateId=311) \n-  [JarvisLabs.ai ](https://jarvislabs.ai/templates/axolotl) \n-  [Latitude.sh ](https://latitude.sh/blueprint/989e0e79-3bf6-41ea-a46b-1f246e309d5c) \n\n\n### Your First Fine-tune  [](https://github.com/axolotl-ai-cloud/axolotl/#your-first-fine-tune) #  Fetch axolotl examples  axolotl fetch examples  #  Or, specify a custom path  axolotl fetch examples --dest path/to/folder  #  Train a model using LoRA  axolotl train examples/llama-3/lora-1b.yml \n\nThat's it! Check out our   [Getting Started Guide ](https://docs.axolotl.ai/docs/getting-started.html)  for a more detailed walkthrough. \n\n## 📚 Documentation  [](https://github.com/axolotl-ai-cloud/axolotl/#-documentation) \n-  [Installation Options ](https://docs.axolotl.ai/docs/installation.html)  - Detailed setup instructions for different environments \n-  [Configuration Guide ](https://docs.axolotl.ai/docs/config-reference.html)  - Full configuration options and examples \n-  [Dataset Loading ](https://docs.axolotl.ai/docs/dataset_loading.html)  - Loading datasets from various sources \n-  [Dataset Guide ](https://docs.axolotl.ai/docs/dataset-formats/)  - Supported formats and how to use them \n-  [Multi-GPU Training ](https://docs.axolotl.ai/docs/multi-gpu.html) \n-  [Multi-Node Training ](https://docs.axolotl.ai/docs/multi-node.html) \n-  [Multipacking ](https://docs.axolotl.ai/docs/multipack.html) \n-  [API Reference ](https://docs.axolotl.ai/docs/api/)  - Auto-generated code documentation \n-  [FAQ ](https://docs.axolotl.ai/docs/faq.html)  - Frequently asked questions \n\n\n## AI Agent Support  [](https://github.com/axolotl-ai-cloud/axolotl/#ai-agent-support) \n\nAxolotl ships with built-in documentation optimized for AI coding agents (Claude Code, Cursor, Copilot, etc.). These docs are bundled with the pip package \u2014 no repo clone needed. #  Show overview and available training methods  axolotl agent-docs  #  Topic-specific references  axolotl agent-docs sft  #  supervised fine-tuning  axolotl agent-docs grpo  #  GRPO online RL  axolotl agent-docs preference_tuning  #  DPO, KTO, ORPO, SimPO  axolotl agent-docs reward_modelling  #  outcome and process reward models  axolotl agent-docs pretraining  #  continual pretraining  axolotl agent-docs --list  #  list all topics #  Dump config schema for programmatic use  axolotl config-schema axolotl config-schema --field adapter \n\nIf you're working with the source repo, agent docs are also available at   `docs/agents/ ` and the project overview is in   `AGENTS.md `. \n\n## 🤝 Getting Help  [](https://github.com/axolotl-ai-cloud/axolotl/#-getting-help) \n- Join our   [Discord community ](https://discord.gg/HhrNrHJPRb)  for support \n- Check out our   [Examples ](https://github.com/axolotl-ai-cloud/axolotl/tree/main/examples/)  directory \n- Read our   [Debugging Guide ](https://docs.axolotl.ai/docs/debugging.html) \n- Need dedicated support? Please contact   [✉️wing@axolotl.ai ](mailto:wing@axolotl.ai)  for options \n\n\n## 🌟 Contributing  [](https://github.com/axolotl-ai-cloud/axolotl/#-contributing) \n\nContributions are welcome! Please see our   [Contributing Guide ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/.github/CONTRIBUTING.md)  for details. \n\n## 📈 Telemetry  [](https://github.com/axolotl-ai-cloud/axolotl/#-telemetry) \n\nAxolotl has opt-out telemetry that helps us understand how the project is being used and prioritize improvements. We collect basic system information, model types, and error rates\u2014never personal data or file paths. Telemetry is enabled by default. To disable it, set AXOLOTL_DO_NOT_TRACK=1. For more details, see our   [telemetry documentation ](https://docs.axolotl.ai/docs/telemetry.html) . \n\n## ❤️ Sponsors  [](https://github.com/axolotl-ai-cloud/axolotl/#️-sponsors) \n\nInterested in sponsoring? Contact us at   [wing@axolotl.ai ](mailto:wing@axolotl.ai) \n\n## 📝 Citing Axolotl  [](https://github.com/axolotl-ai-cloud/axolotl/#-citing-axolotl) \n\nIf you use Axolotl in your research or projects, please cite it as follows: @software { axolotl ,  title  =  { Axolotl: Open Source LLM Post-Training } ,  author  =  { {Axolotl maintainers and contributors} } ,  url  =  { https://github.com/axolotl-ai-cloud/axolotl } ,  license  =  { Apache-2.0 } ,  year  =  { 2023 }  } \n\n## 📜 License  [](https://github.com/axolotl-ai-cloud/axolotl/#-license) \n\nThis project is licensed under the Apache 2.0 License - see the   [LICENSE ](https://github.com/axolotl-ai-cloud/axolotl/blob/main/LICENSE)  file for details. \n\n## About \n\n Go ahead and axolotl questions   [docs.axolotl.ai ](https://docs.axolotl.ai) \n\n### Topics  [ fine-tuning  ](https://github.com/topics/fine-tuning)  [ llm  ](https://github.com/topics/llm) \n\n### Resources  [ Readme  ](https://github.com/axolotl-ai-cloud/axolotl/#readme-ov-file) \n\n### License  [ Apache-2.0 license  ](https://github.com/axolotl-ai-cloud/axolotl/#Apache-2.0-1-ov-file) \n\n### Code of conduct  [ Code of conduct  ](https://github.com/axolotl-ai-cloud/axolotl/#coc-ov-file) \n\n### Contributing  [ Contributing  ](https://github.com/axolotl-ai-cloud/axolotl/#contributing-ov-file) \n\n### Security policy  [ Security policy  ](https://github.com/axolotl-ai-cloud/axolotl/#security-ov-file) \n\n### Citation  Cite this repository  Loading \n\n Something went wrong.  \n\n###  Uh oh!  \n\n\n\nThere was an error while loading.   [Please reload this page ]() . \n\n [Activity ](https://github.com/axolotl-ai-cloud/axolotl/activity)  [Custom properties ](https://github.com/axolotl-ai-cloud/axolotl/custom-properties) \n\n### Stars  [ **11.9k **  stars ](https://github.com/axolotl-ai-cloud/axolotl/stargazers) \n\n### Watchers  [ **57 **  watching ](https://github.com/axolotl-ai-cloud/axolotl/watchers) \n\n### Forks  [ **1.3k **  forks ](https://github.com/axolotl-ai-cloud/axolotl/forks)  [ Report repository  ](https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Faxolotl-ai-cloud%2Faxolotl&report=axolotl-ai-cloud+%28user%29) \n\n##  [Releases  30 ](https://github.com/axolotl-ai-cloud/axolotl/releases)  [v0.16.1  Latest  Apr 2, 2026on Apr 2 ](https://github.com/axolotl-ai-cloud/axolotl/releases/tag/v0.16.1)  [+ 29 releases ](https://github.com/axolotl-ai-cloud/axolotl/releases) \n\n## Sponsor this project \n\n\n\n##  [Packages  0 ](https://github.com/orgs/axolotl-ai-cloud/packages?repo_name=axolotl)  No packages published  \n\n###  Uh oh!  \n\n\n\nThere was an error while loading.   [Please reload this page ]() . \n\n\n\n##  [Contributors  244 ](https://github.com/axolotl-ai-cloud/axolotl/graphs/contributors) \n-  [![@winglian](https://avatars.githubusercontent.com/u/381258?s=64&v=4)](https://github.com/winglian) \n-  [![@NanoCode012](https://avatars.githubusercontent.com/u/9899957?s=64&v=4)](https://github.com/NanoCode012) \n-  [![@djsaunde](https://avatars.githubusercontent.com/u/1245942?s=64&v=4)](https://github.com/djsaunde) \n-  [![@salmanmohammadi](https://avatars.githubusercontent.com/u/25081738?s=64&v=4)](https://github.com/salmanmohammadi) \n-  [![@tmm1](https://avatars.githubusercontent.com/u/2567?s=64&v=4)](https://github.com/tmm1) \n-  [![@ved1beta](https://avatars.githubusercontent.com/u/146507396?s=64&v=4)](https://github.com/ved1beta) \n-  [![@hamelsmu](https://avatars.githubusercontent.com/u/1483922?s=64&v=4)](https://github.com/hamelsmu) \n-  [![@mhenrichsen](https://avatars.githubusercontent.com/u/22191282?s=64&v=4)](https://github.com/mhenrichsen) \n-  [![@maximegmd](https://avatars.githubusercontent.com/u/672982?s=64&v=4)](https://github.com/maximegmd) \n-  [![@github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=64&v=4)](https://github.com/apps/github-actions) \n-  [![@viktoriussuwandi](https://avatars.githubusercontent.com/u/68414300?s=64&v=4)](https://github.com/viktoriussuwandi) \n-  [![@chiragjn](https://avatars.githubusercontent.com/u/10295418?s=64&v=4)](https://github.com/chiragjn) \n-  [![@bursteratom](https://avatars.githubusercontent.com/u/22844540?s=64&v=4)](https://github.com/bursteratom) \n-  [![@casper-hansen](https://avatars.githubusercontent.com/u/27340033?s=64&v=4)](https://github.com/casper-hansen) \n [+ 230 contributors ](https://github.com/axolotl-ai-cloud/axolotl/graphs/contributors) \n\n## Languages \n-  [Python 97.4% ](https://github.com/axolotl-ai-cloud/axolotl/search?l=python) \n-  [Jinja 2.3% ](https://github.com/axolotl-ai-cloud/axolotl/search?l=jinja) \n- Other 0.3% \n\n\n## Footer  [](https://github.com)  © 2026 GitHub, Inc.  \n\n### Footer navigation \n-  [Terms ](https://docs.github.com/site-policy/github-terms/github-terms-of-service) \n-  [Privacy ](https://docs.github.com/site-policy/privacy-policies/github-privacy-statement) \n-  [Security ](https://github.com/security) \n-  [Status ](https://www.githubstatus.com/) \n-  [Community ](https://github.community/) \n-  [Docs ](https://docs.github.com/) \n-  [Contact ](https://support.github.com?tags=dotcom-footer) \n-  Manage cookies  \n-  Do not share my personal information  \n You can\u2019t perform that action at this time.","humanLanguage":"en","authorUrl":"https://github.com/axolotl-ai-cloud","pageUrl":"https://github.com/axolotl-ai-cloud/axolotl","authors":[{"name":"axolotl-ai-cloud","link":"https://github.com/axolotl-ai-cloud"}]}]}