no flash atten + sepcache can use together?

[18140663659]

I have two questions:

1. no flash atten + sepcache can use together?
   2）sepcache can be imported from official transformers which is installed by pip ?

[GaussonTschen]

@18140663659 Hi,

1, Of course, SepCache can be used together with flash atten, see details in https://huggingface.co/transformers-community/sep_cache

2, You can install transformers-4.38.0.post1+sepllm-py3-none-any.whl released by us to import SepCache directly from transformers. For the official transformers, you are recommended to use SepCache through official transformers (4.53.0) by custom_generate="transformers-community/sep_cache". See details in https://huggingface.co/transformers-community/sep_cache

You are strongly recommended to read and follow https://huggingface.co/transformers-community/sep_cache carefully to use SepCache properly and do not worry, it is quite easy to use 😊