AI Model Wars: o1-mini vs gpt-4o-mini
On 20 November 2024 I received an email saying that Open AI’s o1-preview and o1-mini models are available for use via the OpenAI API. I did a brief test via my Open WebUI application on my server to prove I could access and use the o1-mini model. I was able to use the model however first impressions of the model left me quite surprised at the performance and quality of the response. It felt like it was no better than gpt-4o-mini and I was paying a considerable amount more for the tokens. I decided to stay on gpt-4o-mini until I could conduct more thorough testing of the new model to see how it fits my use cases. This day has now come and I have documented my findings in this blog post.
How are we going to test.
Open WebUI has some default suggested prompts which we are going to run against each model. The prompts are:
- What are 5 creative things I could do with my kids’ art? I don’t want to throw them away, but it’s also so much clutter.
- Show me a code snippet of a website’s sticky header in CSS and JavaScript.
- Tell me a random fun fact about the Roman Empire.
We will run these prompts against each model and compare the quality of the response. We will also gather the cost of running these queries from the OpenAI Dashboard. All of these queries will be ran on the same machine using the same version of Open WebUI which is v0.4.4 .
Quality of Response
Keep in mind that the responses generated may not be identical. What we want to focus on is the quality of those responses.
What are 5 creative things I could do with my kids’ art? I don’t want to throw them away, but it’s also so much clutter.
Absolutely, finding creative ways to cherish your kids’ artwork while managing clutter is a wonderful approach. Here are five ideas to help you honor their creations: Create a Rotating Gallery Wall Transform Art into a Photobook Design Custom Gifts and Household Items Make a Digital Slideshow or Screensaver Create a Collage or Mosaic Implementing these ideas can help you celebrate your children’s creativity while keeping your home organized and free from excessive clutter. Choose the methods that best fit your lifestyle and the sentimental value you place on their artwork! It’s wonderful that you want to preserve and celebrate your kids’ art! Here are five creative ideas to help you manage the clutter while showcasing their creativity: Create a Rotating Art Gallery: Designate a wall or a space in your home as a mini art gallery. Use frames, clipboards, or string with clothespins to display a selection of their artwork. Rotate the art regularly, allowing your kids to decide which pieces to feature and keeping the display fresh. Art Book Compilation: Photograph or scan your children’s artwork and create a printed photo book. You can use online services to design and print the book, which will serve as a lovely keepsake and take up much less space than storing physical art. Custom Gift Wrap: Use your kids’ art as unique and personal gift wrap for presents. This not only adds a special touch to gifts but also helps minimize clutter by finding a practical use for the artwork. Art Collage or Quilt: Take portions of different art pieces and create a larger collage or have a sewing service make a quilt out of fabric printed with their art. This way, you condense their creations into one beautiful piece that can be displayed or used. Digital Slideshow: Create a digital slideshow or screensaver using scanned or photographed images of their artwork. This way, you can enjoy their creations on your computer or TV screens without the physical clutter. Consider pairing the slideshow with music for an engaging presentation! These ideas can help you celebrate your children’s creativity while also managing the volume of art they produce.o1-mini response
gpt-4o-mini Response
The o1-mini model does give more detail here breaking down the suggestions into how and the benefits of doing so. The gpt-4o-mini model does not do that and captures the how and the benefit in one paragraph. Despite this difference in structure, both models deliver relatively similar answers.
Show me a code snippet of a website’s sticky header in CSS and JavaScript.
Creating a sticky header ensures that your website’s header remains visible at the top of the viewport as users scroll down the page. You can achieve this using CSS alone with the Below is a complete example demonstrating both CSS and JavaScript approaches to creating a sticky header. The simplest way to create a sticky header is by using the While CSS can handle the basic sticky behavior, JavaScript can add more interactive features, such as changing the header’s appearance after the user scrolls a certain distance. With the above HTML, CSS, and JavaScript: You can see a live demo of the sticky header here. If you need broader browser support or different behavior, you can use Note: When using Feel free to customize the styles and JavaScript logic to fit the specific needs and design of your website! Sure! A sticky header is a common UI feature that remains at the top of the viewport as a user scrolls down the page. Below is a simple example of how you can implement a sticky header using CSS and JavaScript. You can add more content inside o1-mini response
position: sticky property or enhance its functionality with JavaScript for additional effects, such as changing the header’s appearance after scrolling a certain distance.1. HTML Structure
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>Sticky Header Example</title>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<header id="header">
<h1>My Website</h1>
<nav>
<a href="#home">Home</a>
<a href="#about">About</a>
<a href="#contact">Contact</a>
</nav>
</header>
<main>
<section id="home">
<h2>Home</h2>
<p>Content goes here...</p>
</section>
<section id="about">
<h2>About</h2>
<p>More content...</p>
</section>
<section id="contact">
<h2>Contact</h2>
<p>Even more content...</p>
</section>
<!-- Add more sections as needed to enable scrolling -->
</main>
<script src="script.js"></script>
</body>
</html>
2. CSS Styling (
styles.css)Using CSS
position: stickyposition: sticky property. This method requires minimal code and works well in modern browsers./* Basic Reset */
* {
margin: 0;
padding: 0;
box-sizing: border-box;
}
body {
font-family: Arial, sans-serif;
}
/* Header Styling */
header {
position: sticky;
top: 0;
background-color: #333;
color: #fff;
padding: 15px 20px;
z-index: 1000; /* Ensure the header stays on top */
transition: background-color 0.3s, padding 0.3s;
}
/* Navigation Links */
header nav a {
color: #fff;
margin: 0 10px;
text-decoration: none;
}
header nav a:hover {
text-decoration: underline;
}
/* Scrolled Header Styling (for JavaScript Enhancement) */
header.scrolled {
background-color: #222;
padding: 10px 20px;
}
/* Main Content */
main {
padding: 20px;
}
section {
margin-bottom: 100vh; /* To create scrollable content */
}
section h2 {
margin-bottom: 10px;
}
Explanation:
position: sticky: This positions the header relative to its nearest scrolling ancestor and sticks to the top (top: 0) when you scroll past it.z-index: 1000: Ensures the header stays above other content.3. JavaScript Enhancement (
script.js)// Select the header
const header = document.getElementById('header');
// Function to add or remove the 'scrolled' class based on scroll position
function toggleHeaderScrolled() {
if (window.scrollY > 50) { // You can adjust the value as needed
header.classList.add('scrolled');
} else {
header.classList.remove('scrolled');
}
}
// Listen to the scroll event
window.addEventListener('scroll', toggleHeaderScrolled);
Explanation:
toggleHeaderScrolled Function: Adds the scrolled class to the header when the user scrolls down more than 50 pixels, and removes it when the user scrolls back up.4. Final Result
Demo
Browser Support
position: sticky is supported in most modern browsers, including Chrome, Firefox, Edge, and Safari.Alternative Approach: Using
position: fixedposition: fixed. However, this method removes the header from the normal document flow, which might require additional styling adjustments.header {
position: fixed;
top: 0;
width: 100%;
/* Other styles remain the same */
}
position: fixed, ensure that the main content has a top margin equal to the header’s height to prevent it from being hidden behind the fixed header.gpt-4o-mini Response
HTML Structure
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Sticky Header Example</title>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<header id="header">Sticky Header</header>
<main>
<section>
<h1>Welcome to the Sticky Header Example</h1>
<p>Scroll down to see the header stick to the top of the page.</p>
<div class="content">
<!-- Add some dummy content to enable scrolling -->
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nulla ac efficitur nulla. Praesent at velit eget justo rhoncus pulvinar. Sed ac posuere lectus, quis pretium ipsum. Etiam eget felis vitae augue elementum ullamcorper a non tortor.</p>
<p>(Add a lot more content here to make the page scrollable)</p>
<p>...</p> <!-- Extend this with more paragraphs to ensure scrolling -->
</div>
</section>
</main>
<script src="script.js"></script>
</body>
</html>
CSS (styles.css)
body {
font-family: Arial, sans-serif;
margin: 0;
padding: 0;
}
header {
background-color: #333;
color: white;
padding: 10px 20px;
text-align: center;
width: 100%;
transition: top 0.3s;
}
main {
padding: 15px;
}
.content {
height: 1500px; /* Set a height for content to ensure the page is scrollable */
}
.sticky {
position: fixed;
top: 0;
left: 0;
right: 0;
z-index: 1000; /* Ensure the header is on top */
}
JavaScript (script.js)
// Get the header
const header = document.getElementById("header");
// Get the offset position of the navbar
const sticky = header.offsetTop;
// Add the sticky class to the header when you reach its scroll position. Remove "sticky" when you leave the scroll position
function stickHeader() {
if (window.pageYOffset > sticky) {
header.classList.add("sticky");
} else {
header.classList.remove("sticky");
}
}
// When the user scrolls the page, execute stickHeader
window.onscroll = function() {
stickHeader();
};
Explanation
<header> element serves as our sticky header. The <main> section contains the content..sticky class sets the header to fixed positioning at the top of the viewport when scrolling occurs..sticky class to the header when the user scrolls past the top position of the header.<div class="content"> to make the page scrollable and demonstrate the sticky header functionality.
Both models provide valid and usable answers, with the o1-mini model offering greater detail and even presenting an alternative approach that users can adopt if they wish to.
Tell me a random fun fact about the Roman Empire.
Fun Fact about the Roman Empire: The Romans were pioneers in engineering and construction, and one of their most impressive feats was the creation of concrete. Roman concrete, known as opus caementicium, was incredibly durable and was used to build iconic structures like the Pantheon and the Colosseum, many of which still stand today. Interestingly, the secret to their concrete’s longevity lies in its unique mixture, which included volcanic ash. This ingredient helped the concrete withstand the test of time, especially in marine environments, allowing Roman harbors and buildings to endure for over two millennia. Modern scientists have studied Roman concrete to inspire more sustainable and long-lasting building materials today.o1-mini response
gpt-4o-mini Response
I felt the quality of the responses was so similar that it was hard to tell which model generated the response. The key difference being that the o1-mini model generated a header, while the gpt-4o-mini model did not.
Cost
The cost of running the three prompts against the o1-mini came up to $0.063462. The cost of running the exact same prompts again gpt-4o-mini cost $0.00123075. This makes running queries against the o1-mini model approximately 5056% more expensive than gpt-4o-mini. I was quite surprised to see such a significant cost difference, especially considering the comparable responses generated by both models.
Conclusion
According to my findings, the average user would be better off sticking with gpt-4o-mini. According to OpenAI blog post on the o1-mini model it looks like you could potentially benefit more if you’re running STEM related prompts against the model which my test do not cover enough to prove that theory. They did note that the models non-STEM topics knowledge is comparable to gpt-4o-mini.