About the role

AI Agentic Tester

Denver, MO (Remote)

Must-Have Skills

* AI Agentic Testing

* Functional Testing

* Generative AI Testing

* Large Language Models (LLMs)

* AI Agents & Agentic AI

* Retrieval-Augmented Generation (RAG)

* Prompt Engineering Validation

* AI Model Validation & Evaluation

API Testing (Postman, REST APIs, Swagger)

* Python

Test Automation (Selenium / Playwright / PyTest)

AI Evaluation Metrics (Accuracy, Hallucination, Relevance, Consistency)

* End-to-End Workflow Testing

* Multi-Agent Testing

* Azure OpenAI / AWS Bedrock / Gemini

Core Responsibilities

Perform functional testing of AI agent workflows and end-to-end AI applications.

Validate LLM responses for accuracy, relevance, consistency, and hallucination rates.

Test RAG pipelines, prompt engineering, and AI agent orchestration.

Execute API testing using Postman, Swagger, and REST APIs.

Develop and maintain automation scripts using Python, Selenium, Playwright, or PyTest.

Validate multi-agent interactions, tool calling, and workflow execution.

Apply AI testing methodologies and evaluation metrics to ensure response quality.

Work with Azure OpenAI, AWS Bedrock, Gemini, or similar AI platforms to validate AI solutions.

Originally posted on Himalayas

Interview prep

Walk in with sharper answers.

Use this as a quick practice sheet before you speak with the employer.

Mid

Technology & IT API integration Python Remote Collaboration Agentic Tester Mid level

Likely questions

Tell us about work you have done that is close to the AI Agentic Tester role.
How would you approach your first 30 days at Delan Associates?
Which of API integration, Python and Remote Collaboration have you used recently, and what did it help you achieve?
Describe a time you solved a problem without waiting to be told exactly what to do.
How do you handle busy days, changing priorities, or pressure at work?

Prepare before the call

A recent example that proves your experience with API integration, Python and Remote Collaboration.
One short story with a problem, your action, and the result.
Two examples that show the strengths listed on your CV.
A clear reason why this role and company interest you.
Your availability, preferred work style, and salary expectations.

Ask them

What would success look like in the first 90 days?
What are the main problems this hire should help solve?
How does the team give feedback and measure good work?
What does a normal working week look like for this role?

Practice line

I am interested in the AI Agentic Tester role because I can bring practical experience in API integration, Python and Remote Collaboration, learn the team quickly, and contribute to the outcomes Delan Associates needs from this hire.