#dl

Introducing more symmetries in attention

https://github.com/NVIDIA/torch-harmonics

https://neurips.cc/virtual/2025/loc/san-diego/poster/117783

GitHub

GitHub - NVIDIA/torch-harmonics: Differentiable signal processing on the sphere for PyTorch

Differentiable signal processing on the sphere for PyTorch - NVIDIA/torch-harmonics

#dl

Park, Chanwook, Sourav Saha, Jiachen Guo, Hantao Zhang, Xiaoyu Xie, Miguel A. Bessa, Dong Qian, et al. 2025. “Unifying Machine Learning and Interpolation Theory via Interpolating Neural Networks.” Nature Communications 16 (1): 1–12.
https://www.nature.com/articles/s41467-025-63790-8

#dl

A few cool ideas in this model.

Introducing Gemma 3n: The developer guide - Google Developers Blog
https://developers.googleblog.com/en/introducing-gemma-3n-developer-guide/

Googleblog

Google for Developers Blog - News about Web, Mobile, AI and Cloud

Learn how to build with Gemma 3n, a mobile-first architecture, MatFormer technology, Per-Layer Embeddings, and new audio and vision encoders.

#dl

So tensorflow and jax are deprecated in the transformers package.

https://github.com/huggingface/transformers/pull/38758

GitHub

Deprecate TF + JAX by Rocketknight1 · Pull Request #38758 · huggingface/transformers

The time has finally come 🔫 🥃

#dl

Sure, nobody uses it anyways.

#dl

PyTorch Native Architecture Optimization: torchao | PyTorch
https://pytorch.org/blog/pytorch-native-architecture-optimization/

#dl

https://soumith.ch/blog/2024-10-02-training-10k-scale.md.html

#dl

There is this new lib called scale. One could compile CUDA code to use it on AMD GPU.

https://docs.scale-lang.com/manual/how-to-use/


I don't know who is more pissed off, NVidia or AMD.

#dl

This repo is really nice.

yuanchenyang/smalldiffusion: Simple and readable code for training and sampling from diffusion models
https://github.com/yuanchenyang/smalldiffusion

GitHub

GitHub - yuanchenyang/smalldiffusion: Simple and readable code for training and sampling from diffusion models

Simple and readable code for training and sampling from diffusion models - yuanchenyang/smalldiffusion

#dl

Google & USC benchmarked a prompt based forecasting method, and the results are amazing.

Cao D, Jia F, Arik SO, Pfister T, Zheng Y, Ye W, et al. TEMPO: Prompt-based Generative Pre-trained Transformer for time series forecasting. arXiv [cs.LG]. 2023. Available: http://arxiv.org/abs/2310.04948

#dl

I am experimenting with torch 2.0 and searching for potential training time improvements in lightning. The following article provides a very good introduction.

https://lightning.ai/pages/community/tutorial/how-to-speed-up-pytorch-model-training/

Lightning AI

How to Speed Up PyTorch Model Training

Learn how to improve the training performance of your PyTorch model without compromising its accuracy.

#dl

https://github.com/Lightning-AI/lightning/releases/tag/2.0.0

You can compile (torch 2.0) LightningModule now.

import torch
import lightning as L
model = LitModel()
# This will compile forward and {training,validation,test,predict}_step
compiled_model = torch.compile(model)
trainer = L.Trainer()
trainer.fit(compiled_model)

GitHub

Release Lightning 2.0: Fast, Flexible, Stable · Lightning-AI/lightning

Lightning AI is excited to announce the release of Lightning 2.0 ⚡

Highlights
Backward Incompatible Changes

PyTorch
Fabric


Full Changelog

PyTorch
Fabric
App


Contributors

Over the last coupl...

#dl

Hmm I'm talking to ... a paper?

https://github.com/mukulpatnaik/researchgpt

GitHub

GitHub - mukulpatnaik/researchgpt: A LLM based research assistant that allows you to have a conversation with a research paper

A LLM based research assistant that allows you to have a conversation with a research paper - mukulpatnaik/researchgpt