ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way

Abstract

The text-to-video (T2V) generation models, offering convenient visual creation, have recently garnered increasing attention. Despite their substantial potential, the generated videos may present artifacts, including structural implausibility, temporal inconsistency, and a lack of motion, often resulting in near-static video. In this work, we have identified a correlation between the disparity of temporal attention maps across different blocks and the occurrence of temporal inconsistencies. Additionally, we have observed that the energy contained within the temporal attention maps is directly related to the magnitude of motion amplitude in the generated videos. Based on these observations, we present ByTheWay, a training-free method to improve the quality of text-to-video generation without introducing additional parameters, augmenting memory or sampling time. Specifically, ByTheWay is composed of two principal components: 1) Temporal Self-Guidance improves the structural plausibility and temporal consistency of generated videos by reducing the disparity between the temporal attention maps across various decoder blocks. 2) Fourier-based Motion Enhancement enhances the magnitude and richness of motion by amplifying the energy of the map. Extensive experiments demonstrate that ByTheWay significantly improves the quality of text-to-video generation with negligible additional cost.

Author abstract · arXiv abstract · v3

Publication

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2025

Paper and resources

• Paper · arXiv v3 (PDF)
• arXiv:2410.06241
• Google Scholar
• Code · Bujiazi/ByTheWay

Research topics

ByTheWay · Text-to-video generation · Training-free guidance · Temporal attention · Temporal consistency · Motion enhancement · Fourier analysis · Video diffusion

Research problem and approach

Text-to-video models can produce structural artifacts, flicker and nearly static clips. ByTheWay connects cross-block attention-map disagreement with inconsistency and attention-map energy with motion amplitude, then intervenes directly during generation. arXiv abstract · v3

Main contributions

• Introduces Temporal Self-Guidance to coordinate temporal attention across decoder blocks. arXiv abstract · v3
• Uses Fourier-based Motion Enhancement to increase motion magnitude and richness. arXiv abstract · v3

Method comparison

arXiv abstract · v3

Selected results

• Adding ByTheWay to AnimateDiff improves VBench motion smoothness from 0.9474 to 0.9786 and dynamic degree from 0.4073 to 0.5245. FreeInit reaches 0.9713 smoothness but 0.2941 dynamic degree, illustrating the importance of measuring both smoothness and motion extent. Table 2 · VBench, AnimateDiff backbone · arXiv v3
• In paired video-quality comparisons, users prefer ByTheWay outputs in 74.58% of AnimateDiff comparisons and 69.46% of VideoCrafter2 comparisons. These preference percentages are separate from the MLLM-based structure and motion assessments in the same table. Table 1 · human preference study · arXiv v3

Cite this paper

Jiazi Bu, Pengyang Ling, Pan Zhang, Tong Wu, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Dahua Lin, Jiaqi Wang. ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 12999–13008.

@inproceedings{arxiv241006241,
  title     = {{ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way}},
  author    = {Jiazi Bu and Pengyang Ling and Pan Zhang and Tong Wu and Xiaoyi Dong and Yuhang Zang and Yuhang Cao and Dahua Lin and Jiaqi Wang},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2025},
  pages     = {12999--13008},
  url       = {https://openaccess.thecvf.com/content/CVPR2025/html/Bu_ByTheWay_Boost_Your_Text-to-Video_Generation_Model_to_Higher_Quality_in_CVPR_2025_paper.html}
}