Home-Server MachineLearning

Using AI in programming Part 3

To make it short: I’m now using an existing opensource tool for VSCode which can point to my local server. It’s called open copilot and is licensed under Apache 2 which allows commercial use.

Once setting up the url for your local server you can query the extension “cody” with the icon on the left sidebar or within inline code.

Its somewhat usable on my machine.


Using AI in programming Part 2

Here i will describe how to install the code llama modell on one of your VM’s. Problem here is that you usually loose the GPU support because MS is still not able to provide GPU-P for Hyper-V. TY MS at this point.

First prepare a vm, I’m choosing ubuntu srv 22.04.03 LTS. Assigning 8 cores and 64GB of RAM.

Check if git is installed (insert “git –version” in the terminal):

Clone the llama.cpp project, insert the following commands in the terminal (“make” w/o anything is cpu only, you may need to install the gcc compiler):

git clone
cd llama.cpp
sudo apt install build-essential //gcc compiler

Downloading the model can be done with a nice script:

bash <(curl -sSL -m TheBloke/CodeLlama-70B-hf-GGUF --concurrent 8 --storage models/

After downloading the modell, we can start llama.cpp.

./server \
    --model ./models/TheBloke_CodeLlama-70B-hf-GGUF/codellama-70b-hf.Q5_K_M.gguf --host --port 8080 -c 2048

Next step would be to create a plugin for VS e.g. to autofill code from commentblock.


Using AI in programming

This is a an example on how to use AI in your own environment. I’m using the popular LLM Code Llama from Meta which is open source (also for commercial use) and supports C#, my main programming language. Another advantage is that you can use it in your own environment and your most value asset the sourcecode will not leave your network.

Code Llama comes in different sizes, I’m currently using the 70B Model which returns the best result but requires quite a bit of resources. I’m showing you how to self host a tool which runs the llm locally on a central server and write a simple VSCode extension to access the api to query results.

But first lets try to run it locally on my development machine. For this just install ollama (MIT License) and select the model from meta.

It usually gives you a code snipped and an explanation. It works, but it may be a bit inconvenient. In the next post, I’ll show you how to run the modell on your own virtual environment and access it via web or inside your code editor.


Setting up the environment: Jupyter and Colab

I will start experimenting with neuronal networks using Jupyter Notebooks inside the Google Colab envirnonment. This environment brings everything we need to start with machine learning. It is a web based service having an integrated python IDE and has Keras / Tensorflow already included.

When finished with the first examples i will create my own environment to experiment locally.


What can you do with deep learning/machine learning

I already introduced my motivation to start getting into machine learning and now i want to focus on what can you do with machine learning:

  • Image classification
  • Object detection
  • Voice recognition
  • Data science and predictions

How does this work ?

Keras / Tensorflow and other similar tools are using neuronal networks. This networks can be trained based on sample data. When trained you can insert a new sample to the model and predict its outcome.

Allgemein MachineLearning

Machine learning 1

In the last few years i heard a lot about machine learning, deep learning and machine learning. Since i was already planning to do a little bit more of python programming i want to start using it with Keras / TensorFlow to replay some of the beginner examples. I will run the examples on VMs on my new Server.

Why machine learning ?

Its simple: Its a future technology and currently a lot of companies are beginning to use it.

Why starting with TensorFlow ?

I read an article about a free “self-study guide for aspiring machine learning practitioners“. After doing the crash course and with a little bit of additional work it should be possible to aquire a certification.

What is TensorFlow / Keras ?

Tensorflow is an open-source machine learning library from Google and Keras is an abstraction layer / wrapper for Tensorflow and other similar libraries. Tensorflow allows the design of dataflow graphs where your input moves through a series of mathematical operations. Keras provides functionalities for neuronal networks.