GitHub - badaramoni/wave-field-llm: Wave Field AI — a efficient attention architecture for language models

Abstraction: O(N log N) FFT-based attention replacing quadratic dot-product attention

Key points:

• Replaces standard O(N²) dot-product attention with O(N log N) training complexity via FFT convolution; inference is O(1) per token — constant speed regardless of context length
• At 32K context: 21.8x faster throughput and 5.3x less memory than standard attention; supports 128K context where standard attention OOMs
• DCLM CORE benchmark (130M model): 46.8% accuracy vs GPT-2 target of 26.5%
• Runs on consumer GPUs — 80+ tokens/sec on a laptop; supports INT8 compression at 3.1x with quality preserved
• Models trained from 130M to 1.5B parameters; architecture at v9 and actively scaling; patent pending

Connections: Transformers · Large Language Models · Attention Mechanism · Github

Source: https://github.com/badaramoni/wave-field-llm