State-of-the-Art / Neural Networks and Deep Learning

FLM-101B: Training 101 Billion Parameter Language Model with a $100K Budget

24 September 2023

FLM-101B: Training 101 Billion Parameter Language Model with a $100K Budget

Researchers from Beijing University present FLM-101B, an open-source large language model (LLM) with 101 billion parameters trained from scratch with a budget of only $100K. Training LLMs at large scales…

Würstchen: An Open-Source Text-to-Image Model Consuming 16 Times Less GPU than Stable Diffusion 1.4

14 September 2023

Würstchen: An Open-Source Text-to-Image Model Consuming 16 Times Less GPU than Stable Diffusion 1.4

Würstchen is an open text-to-image model that generates images faster than diffusion models like Stable Diffusion while consuming significantly less memory, achieving comparable results. The approach is based on a…

Persimmon-8B: An Open Model with a 16k Token Context, Running on a Single GPU

11 September 2023

Persimmon-8B: An Open Model with a 16k Token Context, Running on a Single GPU

Researchers from Adept have introduced the open-source language model Persimmon-8B with a 16k token context, which is four times larger than the most compact Llama 2 and text-davinci-002 used in…

Falcon 180B: The Largest Open Language Model Surpasses Llama 2 and GPT 3.5

6 September 2023

Falcon 180B: The Largest Open Language Model Surpasses Llama 2 and GPT 3.5

The Institute of Technological Innovations from the UAE has unveiled Falcon 180B, the largest open language model, displacing Llama 2 from the top spot in the rankings of pre-trained open-access…

GigaGAN: Open Source Model Generates 512px Images in Just 0.13 Seconds

1 September 2023

GigaGAN: Open Source Model Generates 512px Images in Just 0.13 Seconds

GigaGAN – an open source model with 1 billion parameters, can generate 512×512 pixel images in just 0.13 seconds, significantly faster than diffuse and autoregressive models. Additionally, researchers have developed…

Code Llama: State-of-the-Art Code Creation Model

28 August 2023

Code Llama: State-of-the-Art Code Creation Model

The Code Llama model is an enhanced version of Llama 2, designed for code generation, completion, and correction. It’s available for free for both commercial and research purposes. Code Llama…

ReLoRA: Method for Enhancing Performance in Training Large Language Models

16 August 2023

ReLoRA: Method for Enhancing Performance in Training Large Language Models

ReLoRA is a technique for training large transformer-based language models using low-rank matrices, aimed at boosting training efficiency. The effectiveness of this method increases with the scale of the models.…

NVIDIA FlexiCubes: Crafting 3D Grids Using Adaptive Parameters

13 August 2023

NVIDIA FlexiCubes: Crafting 3D Grids Using Adaptive Parameters

NVIDIA has introduced FlexiCubes – a method for generating 3D grids of objects through adaptive parameters. This innovation is designed to deliver the highest quality grids, catering to a wide…

Audiocraft: Open Source Library for Music and Sound Generation

4 August 2023

Audiocraft: Open Source Library for Music and Sound Generation

Introducing Audiocraft – a PyTorch library with open-source code, designed for generating music and sound from text. It serves as a powerful tool for deep learning-based audio generation research. Within…

PIGINet: Generating Optimal Sequence of Robot Actions

30 July 2023

PIGINet: Generating Optimal Sequence of Robot Actions

MIT researchers have introduced PIGINet, a neural network designed to teach robots how to navigate through various tasks. PIGINet evaluates potential action sequences based on task descriptions, scene images, and…

Llama 2 and Llama-2-Chat: A New Generation of Open Source Language Models

19 July 2023

Llama 2 and Llama-2-Chat: A New Generation of Open Source Language Models

The new generation of Llama models comprises three large language models, namely Llama 2 with 7, 13, and 70 billion parameters, along with the fine-tuned conversational models Llama-2-Chat 7B, 34B,…

Google Bard Update: Image Processing and New Language Support

16 July 2023

Google Bard Update: Image Processing and New Language Support

Google Bard has undergone an update, expanding its functionality to 46 languages across more than 200 countries, including countries in Europe and Brazil. The latest features include image processing, dialog…

PACGen: Personalized and Controllable Text-to-Image Generation

7 July 2023

PACGen: Personalized and Controllable Text-to-Image Generation

Researchers from the University of Wisconsin-Madison have introduced a text-to-image diffusion model called PACGen (Personalized and Controllable Text-to-Image Generation) for transferring objects from one image to a new scene generated…

DragGAN: Open Source Model for Manipulating GAN-Generated Images

6 July 2023

DragGAN: Open Source Model for Manipulating GAN-Generated Images

Researchers from the Max Planck Institute, MIT, and Google have introduced DragGAN, an innovative approach that allows for seamless manipulation of images generated using Generative Adversarial Networks (GANs). By leveraging…

Microsoft’s Phi-1 Model with 1.3B Parameters Achieves SotA in Code Generation

30 June 2023

Microsoft’s Phi-1 Model with 1.3B Parameters Achieves SotA in Code Generation

Researchers at Microsoft Research have introduced Phi-1, a language model for code generation with just 1.3 billion parameters. This model has achieved a state-of-the-art level of code generation using a…

AudioPaLM: Google’s Multimodal Model for Voice Translation

29 June 2023

AudioPaLM: Google’s Multimodal Model for Voice Translation

Google has introduced AudioPaLM, a large language model for speech processing and generation that combines two Google language models, PaLM-2 and AudioLM, into a multimodal architecture. The model can recognize…

MAGVIT: Open Source Generative Video Transformer 10-in-1

29 June 2023

MAGVIT: Open Source Generative Video Transformer 10-in-1

Researchers from Carnegie Mellon University, Google Research, and the University of Georgia have introduced MAGVIT (Masked Generative Video Transformer), an open-source video generation model. MAGVIT is a unified model that…

Inflection-1: A Powerful Language Model Surpassing GPT-3.5 in Logical Problem Solving

26 June 2023

Inflection-1: A Powerful Language Model Surpassing GPT-3.5 in Logical Problem Solving

Inflection, a startup, officially introduced Inflection-1, a large language model powering the chatbot Pi. Comparable to GPT-3.5, Inflection-1’s size and capabilities match those of ChatGPT. Training took place on “thousands”…

MusicGen: Open Source Neural Network for Generating Music in Any Genre

13 June 2023

MusicGen: Open Source Neural Network for Generating Music in Any Genre

MusicGen is a neural network that generates music based on textual descriptions and melody examples, providing more precise control over the generated output. Researchers conducted extensive empirical research to demonstrate…

ConPLex: Language Model for Drug Development

11 June 2023

ConPLex: Language Model for Drug Development

ConPLex is a language model trained to analyze chemical databases and search for potential drug molecules that interact best with specific target proteins. The model enables the exploration of over…

LIMA: Pretraining Method on 1000 Examples Achieved GPT4-Level Accuracy

31 May 2023

LIMA: Pretraining Method on 1000 Examples Achieved GPT4-Level Accuracy

Language models typically undergo two stages of training: unsupervised pretraining and fine-tuning to specific tasks and user preferences. The novel LIMA method (Less Is More for Alignment) challenges the traditional…