Member-only story
Update (18th Sept 2023): Previously, I had mentioned that Azure is always faster but after testing with the two different version of gpt-3.5 models (0301 and 0613 version), that doesn’t hold true.
Since OpenAI models (GPT-3.5 and GPT-4) are available directly through OpenAI and through the Azure AI service, I wanted to check which one has better performance. So that’s what I did. I did a simple test to figure out the response time for GPT-3.5 and GPT-4 via both providers.
I tested two prompts to test different number of input and generated tokens. Here are the two test cases used for the messages field of the request to the API.
Test case 1
"messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Translate from English to Spanish: This is a test sentence that has to be translated from english to spanish so that we can benchmark OpenAI"}]Test case 2
"messages": [{"role": "system", "content": "You are a helpful assistant."},{"role": "user", "content": "Translate from English to Spanish: The computer is a miraculous scientific invention. It can store data and perform complex calculations in a very short period of time. Its primary characteristics are speed and accuracy. As a result, computers facilitate quick transactions and communication. It’s now common in schools, colleges, libraries, hospitals, offices, and banks. It is used in…
