This note demonstrates how to initiate an endpoint with FastChat for local LLM application in AutoGen. We only use FastChat here for hosting LLMs. However, it could also be used for training, and evaluating LLMs. Preparation Install FastChat with pip. pip3 install "fschat[model_worker,webui]" Some tutorials also recommend to clone FastChat from github, but it seems not work for me. After cloning the repository, I cannot start the server (controller). Now create a directory for your project and download the model checkpoint or import from Hugging Face.

What is Fine-tuning? Fine-tuning is taking a pre-trained model and training at least one internal model parameter (i.e. weight). In the context of LLMs, Fine-tuning means transforming a general-purpose base model into a specialized model for a particular use case. Why Fine-tune? Generally, a smaller (fine-tuned) model can often outperform larger (more expensive) models on the set of tasks on which it was trained. How to Fine-tune? Self-supervised Learning Self-supervised learning consists of training a model based on the inherent structure of the training data.

What is an LLM? LLM is short for Large Language Model. All the LLMs are language models, but not all language models are LLMs. The distinctions between large and ‘regular’ are 2 key properties, quantitatively and quanlitatively. Quantitatively means the number of parameters used in the model is pretty large. Qualitatively means that when a language model becomes large, something remarkble will happen. Zero-shot Learning This terminology means that the model can perform a task even if it has not been explicitly trained to do it.

An Introduction In the past, transitioning a software application from code to production was a painfully slow process. Developers often struggled with the tedious task of managing dependencies while setting up their test and production environments. Docker, a type of container technology, emerged as a solution to alleviate this pain. From Bare Metal to Docker Bare Metal: The applications ran directly on the physical server (Bare Metal). Teams purchased, racked, stacked, powered on, and configured every new machine.

The basics of user experience design Definitions User experience: How a person, the user, feels about interacting with, or experiencing, a product. Product: A good, service or feature. Characteristics of a good user experience Primary characteristics usable Equitable enjoyable useful User experience career Interaction designer: Interaction designers focus on designing the experience of a product and how it functions. Visual designer: Visual designers focus on how a product or technology looks.