Skip to content

EIAS User Guide

A comprehensive guide for researchers using EIAS to collect expert knowledge.

Table of Contents

  1. Overview
  2. Core Concepts
  3. Managing Projects
  4. Creating Interview Agents
  5. Inviting Experts
  6. The Expert Experience
  7. Extracting Insights
  8. Reviewing and Integrating Insights
  9. Best Practices

Overview

EIAS enables you to collect domain expertise asynchronously through AI-powered interviews. Instead of scheduling meetings with experts, you:

  1. Define what you need to learn (uncertainty areas)
  2. Create an AI agent configured to explore those areas
  3. Send experts a link to complete an interview on their own time
  4. Review synthesized insights and integrate valuable knowledge

This approach scales expert elicitation without the scheduling bottleneck.

Core Concepts

Projects

A project represents a research effort with specific goals. Projects contain: - Goals: What you're trying to achieve - Uncertainty Areas: Knowledge gaps to address - Interview Agents: AI agents configured to interview experts - Integrated Knowledge: Approved insights from completed interviews

Uncertainty Areas

An uncertainty area is a specific knowledge gap. Each area includes: - Title: Brief name (e.g., "Clinical Workflow Bottlenecks") - Description: What you need to learn - Priority: High, Medium, or Low - Questions: Specific questions for experts - Status: Open, Partially Addressed, or Resolved

Interview Agents

An interview agent is an AI configured to interview experts about specific uncertainty areas. Agents have: - Focus Areas: Which uncertainty areas to probe - Interview Style: How the agent conducts interviews - System Prompt: Generated instructions for Claude

Invitations

An invitation is a secure link allowing an expert to complete an interview. Invitations: - Require no login from the expert - Expire after a configurable period (default: 7 days) - Can only be used by one expert per link

Insights

An insight is a piece of knowledge extracted from an interview. Each insight has: - Content: The actual insight - Category: Type of knowledge (fact, opinion, recommendation, etc.) - Confidence: How certain the extraction is (high, medium, low) - Source Messages: Which expert responses informed it

Managing Projects

Creating a Project

  1. From the dashboard, click "New Project"
  2. Fill in the form:
  3. Name: Clear, descriptive title
  4. Description: Context and scope
  5. Goals: Key objectives (one per line)
  6. Click "Create Project"

Example:

Name: EHR Integration Study
Description: Understanding how clinicians interact with EHR systems
during patient encounters and identifying friction points.

Goals:
- Document current EHR workflow patterns
- Identify major friction points in clinical documentation
- Gather improvement recommendations from frontline users

Adding Uncertainty Areas

Good uncertainty areas are: - Specific: Address one knowledge gap - Answerable: An expert could provide useful information - Prioritized: Clear importance relative to project goals

Steps:

  1. From your project page, click "Add Uncertainty Area"
  2. Complete the form:
  3. Title: Brief identifier
  4. Description: What you need to learn
  5. Priority: High, Medium, or Low
  6. Questions: Specific questions (optional but recommended)
  7. Click "Save"

Example:

Title: Documentation Time Burden

Description: How much time do clinicians spend on documentation
versus direct patient care, and what factors influence this ratio?

Priority: High

Questions:
- What percentage of your day is spent on documentation?
- Which documentation tasks take the most time?
- What workarounds have you developed to save time?
- How does documentation time vary by patient complexity?

Project Dashboard

Your project page shows: - Summary: Name, description, status, goals - Uncertainty Areas: Knowledge gaps with status - Interview Agents: Configured agents and their status - Recent Sessions: Latest completed interviews

Creating Interview Agents

Understanding Interview Styles

Style Best For Behavior
Exploratory New domains, broad knowledge gaps Open-ended, follows expert's lead, surfaces unexpected insights
Focused Specific questions, validating assumptions Structured, ensures all questions addressed
Validating Confirming hypotheses, cross-checking information Probing, asks for evidence and counterexamples

Creating an Agent

  1. From your project page, click "New Agent"
  2. Configure the agent:
  3. Name: Descriptive identifier
  4. Focus Areas: Select uncertainty areas to cover
  5. Interview Style: Choose based on your needs
  6. Time Estimate: Expected interview duration
  7. Review the generated system prompt (optional)
  8. Click "Create Agent"

System Prompt Generation

EIAS automatically generates a system prompt that includes: - Project context and goals - Selected uncertainty area descriptions - Focus questions from each area - Interview style instructions - Time guidance - PHI prevention guidelines - Best practices (one question at a time, follow-up on responses)

You can view and modify the generated prompt if needed.

Agent Best Practices

  • One agent per expert type: Create separate agents for different expertise areas
  • 3-5 focus areas maximum: Too many areas create unfocused interviews
  • 15-30 minute time estimates: Longer interviews have lower completion rates
  • Test before deploying: Use "Test Interview" to verify agent behavior

Inviting Experts

Creating Invitations

  1. From the agent page, click "Create Invitation"
  2. Enter expert details:
  3. Name: Expert's full name
  4. Role: Their title or expertise
  5. Email: Optional, for your records
  6. Expiration: When the link expires
  7. Click "Create Invitation"
  8. Copy the generated link

Sharing Invitations

Share the link via: - Email: Include context about the project and time commitment - Slack/Teams: Direct message with brief explanation - In person: During a meeting, share for async follow-up

Sample invitation message:

Hi [Expert Name],

I'm researching [topic] and would value your expertise. I've set up
a brief AI-assisted interview (~15 minutes) to capture your insights.

Please complete it at your convenience using this link:
[Interview Link]

The link expires in 7 days. No login required.

Thank you!
[Your Name]

Managing Invitations

The Invitations page shows all active invitations across projects: - Pending: Expert hasn't started - Opened: Expert clicked link but hasn't begun - In Progress: Interview underway - Completed: Interview finished - Expired: Link no longer valid

The Expert Experience

What Experts See

When an expert clicks their invitation link:

  1. Welcome Screen: Project context, time estimate, privacy note
  2. Chat Interface: Conversational AI interview
  3. Completion Screen: Thank you message, optional feedback

Interview Flow

  1. Agent introduces itself and the project context
  2. Agent asks initial question based on focus areas
  3. Expert responds naturally
  4. Agent follows up, adapts based on responses
  5. Agent covers multiple focus areas organically
  6. Interview concludes when areas are addressed or time limit approached

Expert Features

  • No login required: Token-based access
  • Pause anytime: Close browser, return later
  • Session persistence: Full conversation preserved
  • Completion control: Expert decides when to finish
  • Optional feedback: Rate experience and add corrections

Privacy Considerations

The agent is instructed to: - Not collect protected health information (PHI) - Acknowledge concepts without repeating identifiers - Focus on processes and experiences, not specific cases

Extracting Insights

When to Extract

Extract insights after an interview is completed. Wait for: - Session status: Completed - Sufficient conversation depth (typically 8+ exchanges)

Running Extraction

  1. Navigate to the session page (from agent or project)
  2. Review the transcript to understand the conversation
  3. Click "Extract Insights"
  4. Wait for Claude to analyze the transcript (10-30 seconds)
  5. Review the extracted insights

What Gets Extracted

Claude identifies 3-7 insights per interview, each with:

Field Description
Content The insight itself (1-3 sentences)
Category Type of knowledge
Confidence High, Medium, or Low
Source Messages Which expert responses informed it

Insight Categories

Category Description Example
factual_claim Verifiable statement "Most clinicians spend 2-3 hours daily on documentation"
expert_opinion Professional judgment "The current EHR interface prioritizes billing over care"
process_description How things work "Nurses typically pre-chart vitals before the physician enters"
recommendation Suggested action "Implementing voice dictation would significantly reduce documentation time"
warning Potential pitfall "Rushing documentation often leads to copy-paste errors"
opportunity Identified possibility "Mobile documentation during patient encounters is underutilized"
constraint Limitation or requirement "HIPAA compliance requires all documentation within 24 hours"

Reviewing and Integrating Insights

Review Workflow

Each insight starts with status "Pending Review". You can:

Action Result
Integrate Add to project knowledge, marks as integrated
Review Later Keep in queue, come back to it
Discard Remove from queue, not useful

Review Interface

The insight review page shows: - Summary statistics: Total, pending, integrated counts - Filters: By status, category, confidence - Insight cards: Content, metadata, source messages - Reviewer notes: Add context to decisions

Project-Level Review

View all insights for a project at /projects/[id]/insights: - Aggregated across all agents and sessions - Filter by status, category, confidence - Bulk review capabilities

Global Review

View all pending insights at /insights: - Insights from all projects - Quick triage across research efforts - Links to individual sessions for context

Integration Best Practices

  • Review source messages: Verify the insight accurately represents what was said
  • Add reviewer notes: Document why you integrated or discarded
  • Consider confidence: High confidence insights are more reliable
  • Cross-reference: Compare insights across multiple experts

Best Practices

Project Design

  1. Start small: Begin with 2-3 uncertainty areas
  2. Be specific: Vague areas produce vague insights
  3. Prioritize ruthlessly: Focus on high-value knowledge gaps
  4. Iterate: Add areas as you learn from initial interviews

Agent Configuration

  1. Match style to goal: Exploratory for new domains, focused for specific questions
  2. Test extensively: Run test interviews before sending to real experts
  3. Keep focus areas related: Disparate topics create disjointed interviews
  4. Set realistic time estimates: Overpromising frustrates experts

Expert Selection

  1. Target expertise: Match experts to uncertainty areas
  2. Diversify perspectives: Multiple experts reveal different facets
  3. Consider availability: Async interviews work for busy experts
  4. Explain the value: Help experts understand how their input will be used

Insight Quality

  1. Don't over-extract: 3-7 insights per interview is typical
  2. Verify against transcript: Confirm insights match what was said
  3. Triangulate: Same insight from multiple experts increases confidence
  4. Document decisions: Use reviewer notes to track reasoning

Operational Tips

  1. Regular review cadence: Check pending insights weekly
  2. Close the loop: Share how insights were used with experts
  3. Archive completed projects: Keep workspace clean
  4. Export periodically: Backup integrated knowledge

FAQ

How long should interviews be?

15-30 minutes is optimal. Shorter interviews may not capture enough depth; longer ones have lower completion rates.

Can I edit the system prompt?

Yes, you can view and modify the generated system prompt from the agent page. Be careful not to remove safety instructions.

Create a new invitation for the same expert. Old links cannot be reactivated.

Can experts see their own insights?

Not in the current version. Insights are only visible to project owners.

How do I handle conflicting insights?

Document the conflict in reviewer notes. Consider creating a follow-up interview to resolve discrepancies.

Can I export insights?

Currently, you can view insights in the web interface. Export features are planned for a future release.

Next Steps