Overview

The RTX 4090 GPU was successfully installed, along with a new power supply to support the system. We are using Ubuntu 24.04 LTS, which has preinstalled CUDA and NVIDIA drivers, simplifying setup. The system is equipped with 32 GB of RAM and 2 TB SSD, providing sufficient resources for high-performance model development.

Model Testing with RTX 4090

Llama3:8B-Instruct-16fp

The first model we tested was llama3:8B-instruct-16fp (16 GB), running on the RTX 4090. Inference speed was in the milliseconds range due to the GPU’s power. The model consistently achieved an accuracy of 94%+ during experimentation with various prompts.

Example:

• Prompt: “Categorize the following website based on its content: [example website content].”
• Response: “The website belongs to the category: ‘Education’.”

The model performed impressively, delivering accurate results in real-time, making it ideal for our categorization tasks.

Experiment: Meta Zero Shot Model with Llama3

We explored combining the Meta Zero Shot model with Llama3 for website categorization.

Approach:

1. Meta Zero Shot predicted the initial website category.
2. Llama3 was then prompted to verify or correct the predicted category using the website content and the prediction.

Example:

• Predicted Category (Meta Zero Shot): “Entertainment”
• Prompt to Llama3: “Is ‘Entertainment’ the correct category for the following content? [content]. If not, provide the correct category.”
• Response (Llama3): “No, the correct category is ‘Sports’.”

Results:

This combined approach decreased the accuracy of Llama3, possibly due to bias introduced by the initial predicted category. Therefore, we reverted to using Llama3 independently for categorization, achieving higher accuracy.