EFS vs S3 with Fargate: When to Use Each and Why You Need Both

When building containerized applications on AWS Fargate, one of the most common questions developers ask is: “Should I use EFS or S3 for storage?”

The answer is: “You need both, but for different purposes.”

EFS (Elastic File System) and S3 (Simple Storage Service) serve fundamentally different roles in containerized architectures. Understanding when to use EFS vs S3 with Fargate is crucial for building scalable, cost-effective applications.

This comprehensive guide explains:

• EFS vs S3: Key differences and when to use each
• Why Fargate needs EFS: Persistent storage for containers
• How to mount EFS to Fargate: Step-by-step implementation
• Real-world use cases: Practical examples with code
• Best practices: Cost optimization and performance tips

Quick Answer: EFS vs S3 with Fargate

TL;DR: Use EFS for file system operations and persistent container storage. Use S3 for object storage, static assets, and long-term archives.

Key Insight: EFS is a POSIX-compliant file system (works like a network drive you can mount), while S3 is object storage (accessed via API, like a cloud hard drive). They solve fundamentally different problems and are often used together in production applications.

Part 1: Understanding the Problem with Fargate Storage

Why Fargate Containers Need External Storage

AWS Fargate is a serverless container platform that runs containers without managing EC2 instances. However, Fargate containers have a critical limitation that every developer must understand:

⚠️ Fargate containers are ephemeral - when a Fargate task stops, all data in its local filesystem is permanently lost.

This is why EFS (Elastic File System) is essential for Fargate applications that need persistent storage.

The Storage Challenge

1

Container Start → Write Data → Container Stop → ❌ Data Lost

This creates problems for applications that need:

• Persistent file storage (logs, uploads, temporary files)
• Shared data across multiple container instances
• File system operations (file locking, directory traversal, POSIX compliance)

Why Not Just Use Container Volumes?

Fargate supports ephemeral storage (up to 200GB), but:

• ❌ Data is lost when container stops
• ❌ Not shared across containers
• ❌ Limited to single container instance

Solution: Mount external storage using EFS (Elastic File System).

Part 2: EFS vs S3 - Fundamental Differences Explained

Understanding the fundamental differences between EFS and S3 is crucial for making the right storage choice with AWS Fargate.

What is EFS?

Amazon EFS (Elastic File System) is a fully managed network file system (NFS v4.1) that provides:

• ✅ POSIX-compliant file system (works exactly like a local disk)
• ✅ Shared access across multiple Fargate tasks/containers
• ✅ Persistent storage (data survives container restarts and task replacements)
• ✅ Concurrent access (multiple Fargate tasks can read/write simultaneously)
• ✅ File system operations (file locking, directory operations, symlinks, permissions)
• ✅ Mount to Fargate (can be mounted as a volume in ECS task definitions)

EFS is the only way to get persistent, shared file storage in AWS Fargate.

What is S3?

Amazon S3 (Simple Storage Service) is object storage that provides:

• ✅ Unlimited scalability (store petabytes of data)
• ✅ 99.999999999% (11 9’s) durability
• ✅ Cost-effective for large-scale storage
• ✅ Web-accessible (HTTP/HTTPS access)
• ✅ Versioning and lifecycle policies

Key Differences

Part 3: When to Use EFS with Fargate (Use Cases)

Knowing when to use EFS with Fargate is essential for building applications that need persistent file storage.

Use EFS When You Need:

1. Persistent File System Access

Applications that need to read/write files like a local disk:

1
2
3
4
5

# ✅ Works with EFS
with open('/mnt/efs/data/file.txt', 'w') as f:
    f.write('Hello, World!')

# ❌ Doesn't work with S3 (need boto3 API)

Examples:

• Web applications with file uploads
• Log aggregation (write logs to files)
• Temporary file processing
• Configuration files that need to be edited

2. Shared Storage Across Containers

Multiple Fargate tasks need to access the same files:

1

Container 1 → EFS → Shared Files ← EFS ← Container 2

Use Cases:

• Load-balanced web apps: All instances share uploaded files
• CI/CD pipelines: Build artifacts shared across stages
• Content management: Multiple workers process same files
• Distributed processing: Workers read/write shared data

3. File System Operations

Applications that rely on POSIX file system features:

• File locking: Prevent concurrent writes
• Directory operations: Create nested directories, traverse trees
• Symlinks: Create symbolic links
• Permissions: Set file permissions (chmod, chown)

4. Low-Latency File Access

Applications that need fast file I/O:

• Real-time processing: Process files as they arrive
• Database files: Some databases can use file system storage
• Cache files: Fast read/write for caching

Part 4: When to Use S3 with Fargate (Use Cases)

Understanding when to use S3 with Fargate helps you leverage cost-effective object storage for the right scenarios.

Use S3 When You Need:

1. Object Storage

Store and retrieve objects (files, images, videos) via API:

1
2
3
4
5
6

# ✅ S3 usage
import boto3

s3 = boto3.client('s3')
s3.upload_file('local-file.jpg', 'my-bucket', 'images/file.jpg')
s3.download_file('my-bucket', 'images/file.jpg', 'downloaded.jpg')

Examples:

• User-uploaded images/videos
• Static website assets
• Backup files
• Data archives

2. Cost-Effective Long-Term Storage

Store large amounts of data at low cost:

• S3 Standard: $0.023/GB-month
• S3 Glacier: $0.004/GB-month (archival)
• S3 Intelligent-Tiering: Automatic cost optimization

3. Web-Accessible Content

Serve content directly via HTTP/HTTPS:

1

https://my-bucket.s3.amazonaws.com/images/photo.jpg

Use Cases:

• CDN origin (CloudFront)
• Public file sharing
• Static website hosting

4. Data Lake / Analytics

Store structured/unstructured data for analytics:

• Data ingestion: Collect data from multiple sources
• ETL pipelines: Transform and load data
• Analytics: Query with Athena, process with EMR

Part 5: Why Use EFS and S3 Together? (Hybrid Approach)

The most effective Fargate architectures use both EFS and S3 together - here’s why and how.

The Hybrid Approach

Most production applications need both EFS and S3:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

┌─────────────────────────────────────────┐
│         Fargate Container               │
│                                         │
│  ┌──────────────┐    ┌──────────────┐  │
│  │   EFS Mount  │    │  S3 Client   │  │
│  │  /mnt/efs    │    │  (boto3)     │  │
│  └──────┬───────┘    └──────┬───────┘  │
│         │                   │          │
└─────────┼───────────────────┼──────────┘
          │                   │
          ▼                   ▼
    ┌──────────┐        ┌──────────┐
    │   EFS    │        │    S3    │
    │ (Files)  │        │ (Objects)│
    └──────────┘        └──────────┘

Real-World Example: Web Application

Scenario: A web application that handles file uploads

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

# 1. User uploads file → Store in EFS (temporary processing)
uploaded_file.save('/mnt/efs/uploads/temp/file.jpg')

# 2. Process file (resize, validate, etc.)
process_image('/mnt/efs/uploads/temp/file.jpg')

# 3. Move to S3 (permanent storage, web-accessible)
s3.upload_file(
    '/mnt/efs/uploads/temp/file.jpg',
    'my-bucket',
    'images/processed/file.jpg'
)

# 4. Clean up EFS (free space)
os.remove('/mnt/efs/uploads/temp/file.jpg')

Why Both?

• EFS: Fast file system access for processing
• S3: Permanent storage, web access, cost-effective

Common Patterns

Pattern 1: Upload → Process → Archive

1

User Upload → EFS (temp) → Process → S3 (permanent) → Clean EFS

Pattern 2: Log Aggregation

1

Application → EFS (logs) → Log Processor → S3 (archived logs)

Pattern 3: Static Assets

1

Build Process → EFS (build artifacts) → Deploy → S3 (CDN origin)

Part 6: How to Mount EFS to Fargate (Step-by-Step Guide)

This section shows you exactly how to mount EFS to your Fargate containers. Follow these steps to set up persistent storage for your ECS Fargate tasks.

Step 1: Create EFS File System

1
2
3
4
5
6
7
8

# Create EFS file system
aws efs create-file-system \
    --creation-token my-efs-fs \
    --performance-mode generalPurpose \
    --throughput-mode provisioned \
    --provisioned-throughput-mibps 100

# Note the FileSystemId (e.g., fs-0123456789abcdef0)

Step 2: Create Mount Targets

EFS needs mount targets in each subnet where Fargate tasks run:

1
2
3
4
5

# Create mount target in subnet
aws efs create-mount-target \
    --file-system-id fs-0123456789abcdef0 \
    --subnet-id subnet-0123456789abcdef0 \
    --security-groups sg-0123456789abcdef0

Step 3: Configure Security Groups

EFS Security Group (inbound rules):

• Type: NFS
• Port: 2049
• Source: Fargate task security group

Fargate Task Security Group (outbound rules):

• Type: NFS
• Port: 2049
• Destination: EFS security group

Step 4: Create ECS Task Definition

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

{
  "family": "my-fargate-task",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "256",
  "memory": "512",
  "containerDefinitions": [
    {
      "name": "my-container",
      "image": "my-app:latest",
      "mountPoints": [
        {
          "sourceVolume": "efs-volume",
          "containerPath": "/mnt/efs",
          "readOnly": false
        }
      ]
    }
  ],
  "volumes": [
    {
      "name": "efs-volume",
      "efsVolumeConfiguration": {
        "fileSystemId": "fs-0123456789abcdef0",
        "rootDirectory": "/",
        "transitEncryption": "ENABLED",
        "authorizationConfig": {
          "iam": "ENABLED",
          "accessPointId": "fsap-0123456789abcdef0"
        }
      }
    }
  ]
}

Step 5: Deploy Fargate Service

1
2
3
4
5
6
7
8

# Create ECS service with Fargate
aws ecs create-service \
    --cluster my-cluster \
    --service-name my-service \
    --task-definition my-fargate-task \
    --desired-count 2 \
    --launch-type FARGATE \
    --network-configuration "awsvpcConfiguration={subnets=[subnet-xxx],securityGroups=[sg-xxx],assignPublicIp=ENABLED}"

Part 7: Best Practices

EFS Best Practices

1. Use EFS Access Points

Access points provide:

• Isolated file system views
• IAM-based access control
• Automatic directory creation

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

{
  "name": "my-access-point",
  "fileSystemId": "fs-0123456789abcdef0",
  "posixUser": {
    "uid": 1000,
    "gid": 1000
  },
  "rootDirectory": {
    "path": "/my-app",
    "creationInfo": {
      "ownerUid": 1000,
      "ownerGid": 1000,
      "permissions": "755"
    }
  }
}

2. Enable Encryption in Transit

Always enable transitEncryption: ENABLED for security:

1
2
3
4
5
6

{
  "efsVolumeConfiguration": {
    "transitEncryption": "ENABLED",
    "transitEncryptionPort": 2049
  }
}

3. Choose Right Performance Mode

• generalPurpose: Most applications (low latency)
• maxIO: High throughput, higher latency

4. Monitor EFS Metrics

Key CloudWatch metrics:

• ClientConnections: Number of clients connected
• DataReadIOBytes: Data read from EFS
• DataWriteIOBytes: Data written to EFS
• PercentIOLimit: I/O performance limit

S3 Best Practices

1. Use Lifecycle Policies

Automatically move data to cheaper storage:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

{
  "Rules": [
    {
      "Id": "Move to Glacier",
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ]
    }
  ]
}

2. Enable Versioning

Protect against accidental deletion:

1
2
3

aws s3api put-bucket-versioning \
    --bucket my-bucket \
    --versioning-configuration Status=Enabled

3. Use S3 Transfer Acceleration

For faster uploads from distant locations:

1
2
3
4
5
6
7

s3.upload_file(
    'file.jpg',
    'my-bucket',
    'images/file.jpg',
    ExtraArgs={'ServerSideEncryption': 'AES256'},
    Config=TransferConfig(use_threads=True)
)

Part 8: Cost Comparison

EFS Costs

Storage: ~$0.30/GB-month (General Purpose) Throughput:

• Bursting: Included (up to baseline)
• Provisioned: $0.05/MiBps-month

Example: 100GB, 100 MiBps throughput

• Storage: 100GB × $0.30 = $30/month
• Throughput: 100 MiBps × $0.05 = $5/month
• Total: ~$35/month

S3 Costs

Storage: ~$0.023/GB-month (Standard) Requests:

• PUT: $0.005 per 1,000 requests
• GET: $0.0004 per 1,000 requests

Example: 100GB, 1M PUT, 10M GET requests

• Storage: 100GB × $0.023 = $2.30/month
• PUT: 1,000 × $0.005 = $5/month
• GET: 10,000 × $0.0004 = $4/month
• Total: ~$11.30/month

Key Insight: S3 is cheaper for storage, but EFS is necessary for file system access.

Part 9: Common Use Cases

Use Case 1: Web Application with File Uploads

Architecture:

1

User → ALB → Fargate (multiple tasks) → EFS (shared uploads) → S3 (permanent storage)

Flow:

1. User uploads file via web app
2. File stored in EFS (/mnt/efs/uploads/)
3. Background worker processes file
4. Processed file moved to S3
5. EFS temp file deleted

Use Case 2: CI/CD Pipeline

Architecture:

1

Git Push → CodeBuild → EFS (build artifacts) → S3 (deployment packages)

Flow:

1. CodeBuild runs in Fargate
2. Build artifacts stored in EFS
3. Artifacts packaged and uploaded to S3
4. Deployment reads from S3

Use Case 3: Log Aggregation

Architecture:

1

Application → EFS (logs) → Log Processor → S3 (archived logs) → Athena (query)

Flow:

1. Applications write logs to EFS
2. Log processor aggregates logs
3. Aggregated logs archived to S3
4. Analytics queries run on S3 with Athena

Part 10: Troubleshooting

Common EFS Issues

Issue 1: “Permission Denied” Errors

Solution: Check EFS access point permissions and POSIX user:

1
2
3
4
5

# Verify access point
aws efs describe-access-points --file-system-id fs-xxx

# Check container user
docker exec -it container id

Issue 2: Slow Performance

Solution:

• Check performance mode (use generalPurpose for low latency)
• Monitor CloudWatch metrics
• Consider provisioned throughput for consistent performance

Issue 3: Mount Timeout

Solution:

• Verify security groups allow NFS (port 2049)
• Check mount targets are in correct subnets
• Ensure DNS resolution works in VPC

Common S3 Issues

Issue 1: “Access Denied” Errors

Solution: Check IAM permissions:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::my-bucket/*"
    }
  ]
}

Issue 2: Slow Uploads

Solution:

• Use multipart uploads for large files
• Enable S3 Transfer Acceleration
• Use parallel uploads with boto3

Frequently Asked Questions (FAQ)

Q1: Can I use S3 instead of EFS for file system access with Fargate?

A: No. S3 cannot replace EFS for file system access. S3 is object storage accessed via REST API, not a POSIX-compliant file system. Applications that need file system operations (file locking, directory traversal, symlinks, etc.) require EFS. You cannot mount S3 as a filesystem in Fargate containers.

Q2: Is EFS more expensive than S3 for Fargate storage?

A: Yes, EFS is more expensive than S3 (~$0.30/GB-month vs ~$0.023/GB-month), but they serve different purposes. EFS provides POSIX-compliant file system functionality that S3 cannot. Best practice: Use EFS for active file operations and temporary storage in Fargate containers, and S3 for long-term storage, backups, and static assets.

Q3: Can multiple Fargate tasks share the same EFS file system?

A: Yes! Multiple Fargate tasks can mount and share the same EFS file system. This is one of EFS’s key features and is essential for load-balanced applications where all container instances need access to the same files (e.g., uploaded files, shared configuration, logs).

Q4: Do I need both EFS and S3 with Fargate?

A: Most production applications use both EFS and S3 for different purposes:

• EFS: Active file operations, shared storage across Fargate tasks, temporary files, application logs
• S3: Permanent storage, backups, static assets, data archives, web-accessible content

Using both allows you to leverage the strengths of each: EFS for fast file system access and S3 for cost-effective long-term storage.

Q5: Can I mount S3 as a file system in Fargate?

A: Technically yes (using tools like s3fs-fuse), but it’s not recommended for production Fargate deployments. S3 is object storage accessed via API, not a native file system. Performance will be poor, and you’ll lose POSIX features. Use EFS for file system needs in Fargate containers.

Q6: What’s the latency difference between EFS and S3 for Fargate?

A:

• EFS: Low latency (milliseconds) - similar to local disk, optimized for file system operations
• S3: Higher latency (tens to hundreds of milliseconds) - REST API calls over network, not suitable for real-time file operations

For applications requiring fast file I/O in Fargate containers, EFS is the clear choice.

Q7: How do I backup EFS data from Fargate?

A: Use AWS Backup (recommended) or manually copy data to S3:

1
2
3
4

# Manual backup: EFS to S3
aws s3 sync /mnt/efs s3://my-backup-bucket/efs-backup/

# Or use AWS Backup service for automated backups

This is a perfect example of using both EFS and S3 together: EFS for active storage in Fargate, S3 for backups.

Conclusion: EFS vs S3 with Fargate - Key Takeaways

EFS and S3 serve fundamentally different purposes in AWS Fargate containerized applications:

Use EFS with Fargate when you need:

• ✅ POSIX-compliant file system access (read/write files like local disk)
• ✅ Persistent storage that survives container restarts
• ✅ Shared storage across multiple Fargate tasks
• ✅ Low-latency file operations (milliseconds)
• ✅ Mount as volume in ECS task definitions

Use S3 with Fargate when you need:

• ✅ Object storage (accessed via REST API)
• ✅ Cost-effective long-term storage (~$0.023/GB-month)
• ✅ Web-accessible content (HTTP/HTTPS)
• ✅ Data lake / analytics (Athena, EMR)
• ✅ Backup destination for EFS data

The Bottom Line

Most production Fargate applications use both EFS and S3:

• EFS for active file operations, shared storage, and temporary files
• S3 for permanent storage, backups, static assets, and archives

Understanding when to use EFS vs S3 with Fargate is crucial for building scalable, cost-effective containerized applications on AWS. Remember: EFS is for file systems, S3 is for objects - and they work best together.

References

• AWS EFS Documentation
• AWS S3 Documentation
• AWS Fargate Documentation
• EFS with ECS Best Practices