Skip to content

Agentic Framework Evaluation — CrewAI vs DeepAgents

Status: Draft (Phase 1 complete; Phase 2 benchmarks pending) Date: 2026-04-20 Source: docs/Smackz-Agentic-Framework-Evaluation.md Related projects: Agentic Workflows Roadmap #18, Brand Intelligence #58, SEO Monitoring Agent #56, SEO Keyword Agent #53

Problem

Two parallel prototypes in Agent-prototype implement the same restaurant competitive-analysis workflow (scout → reviewer → analyst) against Google Places + OpenRouter — one in CrewAI, one in DeepAgents (a thin LangGraph wrapper). A decision is needed on which framework SMACKZ adopts for the agents planned across roadmap projects #18, #40–44, #53, #56, and #58.

Recommendation

Primary framework: DeepAgents (LangGraph-based) for autonomous, operational, reactive, and chat-style agents (9.0/10 weighted vs. CrewAI 5.2/10).

Secondary: CrewAI retained for content-generation crews where persona-driven YAML config genuinely improves output quality (Brand Intelligence #58, Social Media Post Automation #59).

Scoring Summary

Criterion (Weight) CrewAI DeepAgents
No vendor lock-in (15%) 4/10 9/10
Streaming for chat (15%) 5/10 10/10
Checkpointing (15%) 3/10 10/10
Human-in-the-loop (10%) 5/10 9/10
MCP integration (10%) 6/10 9/10
Dynamic routing (10%) 4/10 9/10
Role-play workflows (10%) 10/10 7/10
Migration from raw SDK (5%) 4/10 8/10
Observability (5%) 7/10 9/10
Ecosystem (5%) 7/10 9/10

Key Decisions

  • DeepAgents is a thin wrapper over OSS LangGraph — native streaming via .astream(), first-class checkpointing (Postgres / Redis / SQLite), HITL via interrupt/resume in the graph model, and official langchain-mcp-adapters integration.
  • CrewAI's proprietary @CrewBase / Crew / Process / LLM abstractions create vendor lock-in; streaming was added recently and is limited to final agent output. No native checkpoint store — designed around kickoff() → finish.
  • CrewAI's YAML-config sweet spot is persona-driven role-play crews; DeepAgents-style dict subagents are more flexible for everything else.
  • Both frameworks remain transparent on model selection (OpenRouter handles provider routing).

Scope

In scope: Framework comparison for agents across 9 roadmap projects.

Out of scope: Raw LLM calls, single-tool-call classification, cron-only jobs, DAST/RASP.

Phase 2 Exit Criteria

  • Streaming benchmarks (FWK-05).
  • HITL approval demo (FWK-06).
  • Per-agent + dynamic model routing (MDL-06/07).
  • MCP tool integration validated.
  • Migration plan for Phase 3 agents.

Pointers

  • Prototype repo: Smackz-LLC/Agent-prototype
  • CrewAI prototype: Agent-prototype/crewAI/
  • DeepAgents prototype: Agent-prototype/deepAgents/