Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining logo
AI Tool Profile

Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining

It motivates us to develop a technique to evaluate true loss changes without retraining, with which channels to prune can be selected more reliably and confidently.

Website
github.com
Pricing model
Free
Price start
Contact for pricing

Description of Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining

GitHub Link

The GitHub link is https://github.com/hrcheng1066/ifso

Introduce

The GitHub repository "IFSO" presents an approach for second-order channel pruning using influence functions. The method enables evaluating true loss changes without the need for retraining. The repository provides instructions to set up the environment, download the code, and replace certain files. It outlines steps for pre-training, pruning, and fine-tuning. The repository also acknowledges the contributions of related codes that served as the basis for this work.

Content


Alternatives & Similar Tools

LongLLaMA-handle very long text contexts, up to 256,000 tokens logo

LongLLaMA is a large language model designed to handle very long text contexts, up to 256,000 tokens. It's based on OpenLLaMA and uses a technique called Focused Transformer (FoT) for training. The repository provides a smaller 3B version of LongLLaMA for free use. It can also be used as a replacement for LLaMA models with shorter contexts.

Compare Influence Function Based Second-Order Channel Pruning-Evaluating True Loss Changes For Pruning Is Possible Without Retraining

Quick compare routes for nearby alternatives.

All compare routes →