Is AI smarter than a certified landscaper?
It depends on what you ask it
Sep 30, 2024
Artificial Intelligence (AI) has quickly become the most talked about technology of the last few years. We have all been amazed in seeing how chatbots like OpenAI’s ChatGPT can answer questions, write documents and perform many other tasks that we thought only (super smart) humans could do. At Duranta we are big believers in AI, and we use it extensively in our app to save you 3 hours per day.
But we were curious to know: for all of its exciting promise, how well would AI perform on a landscaping certification tests? Would it know what 24-4-4 means on a bag of fertilizer, or understand how best to prune a tree?
The experiment
For our experiment we chose the International Society of Arboriculture (ISA) Certified Arborist Exam, which some consider to be the hardest landscaping exam. For our AI chatbot, we chose Perplexity.ai’s pro version. We used a publicly available practice exam, with an answer key to grade Perplexity.ai’s performance. The publicly available exam was a practice exam so the results might be a little bit different from if we were testing the AI against the real live exam.
ISA Certified Arborist exam description
The ISA Certified Arborist exam has ten unique knowledge areas, ranging from soil management to safe work practices. It is a 200 question multiple choice exam, with 3.5 hours (210 minutes) to answer all the questions. So a test taker has just over a minute for each question (gulp).
The questions have different weightings, meaning that some questions count for more points than others. This means that the passing grade changes from year to year, but usually a score of 70-76% should be enough to pass.
The results
Perplexity.ai comfortably passed the exam with a score of 79.5%. And it did the entire exam in less than a minute!
The AI bot passed the exam in less than a minute with a score of 79.5%, however its performance was uneven
But that isn’t the end of the story: what we found really surprising is how AI scored in the various knowledge areas:
Digging deeper
Perplexity.ai got it’s best grades on the factual questions, specifically, questions such as tree identification and selection. This makes sense: there is a ton of information about these subjects on the internet, and most of it is pretty accurate from our experience (who hasn’t used a plant identification app?).
On the other hand, Perplexity.ai failed questions that required field context and experience: it logged its lowest scores in soil management, installation and establishment, and tree protection.
The AI bot failed questions that require field context and experience
Experience AND intelligence
The takeaway from all this is that AI is a tremendous technology, it passed the hardest landscaping test in less than a minute. But it isn’t perfect, it failed questions that required experience and specialized knowledge. And if you had followed it blindly while taking care of your trees, you could have a very sorry (and non code-compliant) landscape.
So for now, until the bots get even better, make sure to only use Artificial Intelligence as a supplement to your own experience!
Artificial Intelligence (AI) has quickly become the most talked about technology of the last few years. We have all been amazed in seeing how chatbots like OpenAI’s ChatGPT can answer questions, write documents and perform many other tasks that we thought only (super smart) humans could do. At Duranta we are big believers in AI, and we use it extensively in our app to save you 3 hours per day.
But we were curious to know: for all of its exciting promise, how well would AI perform on a landscaping certification tests? Would it know what 24-4-4 means on a bag of fertilizer, or understand how best to prune a tree?
The experiment
For our experiment we chose the International Society of Arboriculture (ISA) Certified Arborist Exam, which some consider to be the hardest landscaping exam. For our AI chatbot, we chose Perplexity.ai’s pro version. We used a publicly available practice exam, with an answer key to grade Perplexity.ai’s performance. The publicly available exam was a practice exam so the results might be a little bit different from if we were testing the AI against the real live exam.
ISA Certified Arborist exam description
The ISA Certified Arborist exam has ten unique knowledge areas, ranging from soil management to safe work practices. It is a 200 question multiple choice exam, with 3.5 hours (210 minutes) to answer all the questions. So a test taker has just over a minute for each question (gulp).
The questions have different weightings, meaning that some questions count for more points than others. This means that the passing grade changes from year to year, but usually a score of 70-76% should be enough to pass.
The results
Perplexity.ai comfortably passed the exam with a score of 79.5%. And it did the entire exam in less than a minute!
The AI bot passed the exam in less than a minute with a score of 79.5%, however its performance was uneven
But that isn’t the end of the story: what we found really surprising is how AI scored in the various knowledge areas:
Digging deeper
Perplexity.ai got it’s best grades on the factual questions, specifically, questions such as tree identification and selection. This makes sense: there is a ton of information about these subjects on the internet, and most of it is pretty accurate from our experience (who hasn’t used a plant identification app?).
On the other hand, Perplexity.ai failed questions that required field context and experience: it logged its lowest scores in soil management, installation and establishment, and tree protection.
The AI bot failed questions that require field context and experience
Experience AND intelligence
The takeaway from all this is that AI is a tremendous technology, it passed the hardest landscaping test in less than a minute. But it isn’t perfect, it failed questions that required experience and specialized knowledge. And if you had followed it blindly while taking care of your trees, you could have a very sorry (and non code-compliant) landscape.
So for now, until the bots get even better, make sure to only use Artificial Intelligence as a supplement to your own experience!