initial commit
This commit is contained in:
491
docs/user-guide/database/crud.md
Normal file
491
docs/user-guide/database/crud.md
Normal file
@ -0,0 +1,491 @@
|
||||
# CRUD Operations
|
||||
|
||||
This guide covers all CRUD (Create, Read, Update, Delete) operations available in the FastAPI Boilerplate using FastCRUD, a powerful library that provides consistent and efficient database operations.
|
||||
|
||||
## Overview
|
||||
|
||||
The boilerplate uses [FastCRUD](https://github.com/igorbenav/fastcrud) for all database operations. FastCRUD provides:
|
||||
|
||||
- **Consistent API** across all models
|
||||
- **Type safety** with generic type parameters
|
||||
- **Automatic pagination** support
|
||||
- **Advanced filtering** and joining capabilities
|
||||
- **Soft delete** support
|
||||
- **Optimized queries** with selective field loading
|
||||
|
||||
## CRUD Class Structure
|
||||
|
||||
Each model has a corresponding CRUD class that defines the available operations:
|
||||
|
||||
```python
|
||||
# src/app/crud/crud_users.py
|
||||
from fastcrud import FastCRUD
|
||||
from app.models.user import User
|
||||
from app.schemas.user import (
|
||||
UserCreateInternal, UserUpdate, UserUpdateInternal,
|
||||
UserDelete, UserRead
|
||||
)
|
||||
|
||||
CRUDUser = FastCRUD[
|
||||
User, # Model class
|
||||
UserCreateInternal, # Create schema
|
||||
UserUpdate, # Update schema
|
||||
UserUpdateInternal, # Internal update schema
|
||||
UserDelete, # Delete schema
|
||||
UserRead # Read schema
|
||||
]
|
||||
crud_users = CRUDUser(User)
|
||||
```
|
||||
|
||||
## Read Operations
|
||||
|
||||
### Get Single Record
|
||||
|
||||
Retrieve a single record by any field:
|
||||
|
||||
```python
|
||||
# Get user by ID
|
||||
user = await crud_users.get(db=db, id=user_id)
|
||||
|
||||
# Get user by username
|
||||
user = await crud_users.get(db=db, username="john_doe")
|
||||
|
||||
# Get user by email
|
||||
user = await crud_users.get(db=db, email="john@example.com")
|
||||
|
||||
# Get with specific fields only
|
||||
user = await crud_users.get(
|
||||
db=db,
|
||||
schema_to_select=UserRead, # Only select fields defined in UserRead
|
||||
id=user_id,
|
||||
)
|
||||
```
|
||||
|
||||
**Real usage from the codebase:**
|
||||
|
||||
```python
|
||||
# From src/app/api/v1/users.py
|
||||
db_user = await crud_users.get(
|
||||
db=db,
|
||||
schema_to_select=UserRead,
|
||||
username=username,
|
||||
is_deleted=False,
|
||||
)
|
||||
```
|
||||
|
||||
### Get Multiple Records
|
||||
|
||||
Retrieve multiple records with filtering and pagination:
|
||||
|
||||
```python
|
||||
# Get all users
|
||||
users = await crud_users.get_multi(db=db)
|
||||
|
||||
# Get with pagination
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
offset=0, # Skip first 0 records
|
||||
limit=10, # Return maximum 10 records
|
||||
)
|
||||
|
||||
# Get with filtering
|
||||
active_users = await crud_users.get_multi(
|
||||
db=db,
|
||||
is_deleted=False, # Filter condition
|
||||
offset=compute_offset(page, items_per_page),
|
||||
limit=items_per_page
|
||||
)
|
||||
```
|
||||
|
||||
**Pagination response structure:**
|
||||
|
||||
```python
|
||||
{
|
||||
"data": [
|
||||
{"id": 1, "username": "john", "email": "john@example.com"},
|
||||
{"id": 2, "username": "jane", "email": "jane@example.com"}
|
||||
],
|
||||
"total_count": 25,
|
||||
"has_more": true,
|
||||
"page": 1,
|
||||
"items_per_page": 10
|
||||
}
|
||||
```
|
||||
|
||||
### Check Existence
|
||||
|
||||
Check if a record exists without fetching it:
|
||||
|
||||
```python
|
||||
# Check if user exists
|
||||
user_exists = await crud_users.exists(db=db, email="john@example.com")
|
||||
# Returns True or False
|
||||
|
||||
# Check if username is available
|
||||
username_taken = await crud_users.exists(db=db, username="john_doe")
|
||||
```
|
||||
|
||||
**Real usage example:**
|
||||
|
||||
```python
|
||||
# From src/app/api/v1/users.py - checking before creating
|
||||
email_row = await crud_users.exists(db=db, email=user.email)
|
||||
if email_row:
|
||||
raise DuplicateValueException("Email is already registered")
|
||||
```
|
||||
|
||||
### Count Records
|
||||
|
||||
Get count of records matching criteria:
|
||||
|
||||
```python
|
||||
# Count all users
|
||||
total_users = await crud_users.count(db=db)
|
||||
|
||||
# Count active users
|
||||
active_count = await crud_users.count(db=db, is_deleted=False)
|
||||
|
||||
# Count by specific criteria
|
||||
admin_count = await crud_users.count(db=db, is_superuser=True)
|
||||
```
|
||||
|
||||
## Create Operations
|
||||
|
||||
### Basic Creation
|
||||
|
||||
Create new records using Pydantic schemas:
|
||||
|
||||
```python
|
||||
# Create user
|
||||
user_data = UserCreateInternal(
|
||||
username="john_doe",
|
||||
email="john@example.com",
|
||||
hashed_password="hashed_password_here"
|
||||
)
|
||||
|
||||
created_user = await crud_users.create(db=db, object=user_data)
|
||||
```
|
||||
|
||||
**Real creation example:**
|
||||
|
||||
```python
|
||||
# From src/app/api/v1/users.py
|
||||
user_internal_dict = user.model_dump()
|
||||
user_internal_dict["hashed_password"] = get_password_hash(password=user_internal_dict["password"])
|
||||
del user_internal_dict["password"]
|
||||
|
||||
user_internal = UserCreateInternal(**user_internal_dict)
|
||||
created_user = await crud_users.create(db=db, object=user_internal)
|
||||
```
|
||||
|
||||
### Create with Relationships
|
||||
|
||||
When creating records with foreign keys:
|
||||
|
||||
```python
|
||||
# Create post for a user
|
||||
post_data = PostCreateInternal(
|
||||
title="My First Post",
|
||||
content="This is the content of my post",
|
||||
created_by_user_id=user.id # Foreign key reference
|
||||
)
|
||||
|
||||
created_post = await crud_posts.create(db=db, object=post_data)
|
||||
```
|
||||
|
||||
## Update Operations
|
||||
|
||||
### Basic Updates
|
||||
|
||||
Update records by any field:
|
||||
|
||||
```python
|
||||
# Update user by ID
|
||||
update_data = UserUpdate(email="newemail@example.com")
|
||||
await crud_users.update(db=db, object=update_data, id=user_id)
|
||||
|
||||
# Update by username
|
||||
await crud_users.update(db=db, object=update_data, username="john_doe")
|
||||
|
||||
# Update multiple fields
|
||||
update_data = UserUpdate(
|
||||
email="newemail@example.com",
|
||||
profile_image_url="https://newimage.com/photo.jpg"
|
||||
)
|
||||
await crud_users.update(db=db, object=update_data, id=user_id)
|
||||
```
|
||||
|
||||
### Conditional Updates
|
||||
|
||||
Update with validation:
|
||||
|
||||
```python
|
||||
# From real endpoint - check before updating
|
||||
if values.username != db_user.username:
|
||||
existing_username = await crud_users.exists(db=db, username=values.username)
|
||||
if existing_username:
|
||||
raise DuplicateValueException("Username not available")
|
||||
|
||||
await crud_users.update(db=db, object=values, username=username)
|
||||
```
|
||||
|
||||
### Bulk Updates
|
||||
|
||||
Update multiple records at once:
|
||||
|
||||
```python
|
||||
# Update all users with specific criteria
|
||||
update_data = {"is_active": False}
|
||||
await crud_users.update(db=db, object=update_data, is_deleted=True)
|
||||
```
|
||||
|
||||
## Delete Operations
|
||||
|
||||
### Soft Delete
|
||||
|
||||
For models with soft delete fields (like User, Post):
|
||||
|
||||
```python
|
||||
# Soft delete - sets is_deleted=True, deleted_at=now()
|
||||
await crud_users.delete(db=db, username="john_doe")
|
||||
|
||||
# The record stays in the database but is marked as deleted
|
||||
user = await crud_users.get(db=db, username="john_doe", is_deleted=True)
|
||||
```
|
||||
|
||||
### Hard Delete
|
||||
|
||||
Permanently remove records from the database:
|
||||
|
||||
```python
|
||||
# Permanently delete from database
|
||||
await crud_users.db_delete(db=db, username="john_doe")
|
||||
|
||||
# The record is completely removed
|
||||
```
|
||||
|
||||
**Real deletion example:**
|
||||
|
||||
```python
|
||||
# From src/app/api/v1/users.py
|
||||
# Regular users get soft delete
|
||||
await crud_users.delete(db=db, username=username)
|
||||
|
||||
# Superusers can hard delete
|
||||
await crud_users.db_delete(db=db, username=username)
|
||||
```
|
||||
|
||||
## Advanced Operations
|
||||
|
||||
### Joined Queries
|
||||
|
||||
Get data from multiple related tables:
|
||||
|
||||
```python
|
||||
# Get posts with user information
|
||||
posts_with_users = await crud_posts.get_multi_joined(
|
||||
db=db,
|
||||
join_model=User,
|
||||
join_on=Post.created_by_user_id == User.id,
|
||||
schema_to_select=PostRead,
|
||||
join_schema_to_select=UserRead,
|
||||
join_prefix="user_"
|
||||
)
|
||||
```
|
||||
|
||||
Result structure:
|
||||
```python
|
||||
{
|
||||
"id": 1,
|
||||
"title": "My Post",
|
||||
"content": "Post content",
|
||||
"user_id": 123,
|
||||
"user_username": "john_doe",
|
||||
"user_email": "john@example.com"
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Filtering
|
||||
|
||||
Advanced filtering with SQLAlchemy expressions:
|
||||
|
||||
```python
|
||||
from sqlalchemy import and_, or_
|
||||
|
||||
# Complex filters
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
filter_criteria=[
|
||||
and_(
|
||||
User.is_deleted == False,
|
||||
User.created_at > datetime(2024, 1, 1)
|
||||
)
|
||||
]
|
||||
)
|
||||
```
|
||||
|
||||
### Optimized Field Selection
|
||||
|
||||
Select only needed fields for better performance:
|
||||
|
||||
```python
|
||||
# Only select id and username
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
schema_to_select=UserRead, # Use schema to define fields
|
||||
limit=100
|
||||
)
|
||||
|
||||
# Or specify fields directly
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
schema_to_select=["id", "username", "email"],
|
||||
limit=100
|
||||
)
|
||||
```
|
||||
|
||||
## Practical Examples
|
||||
|
||||
### Complete CRUD Workflow
|
||||
|
||||
Here's a complete example showing all CRUD operations:
|
||||
|
||||
```python
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from app.crud.crud_users import crud_users
|
||||
from app.schemas.user import UserCreateInternal, UserUpdate, UserRead
|
||||
|
||||
async def user_management_example(db: AsyncSession):
|
||||
# 1. CREATE
|
||||
user_data = UserCreateInternal(
|
||||
username="demo_user",
|
||||
email="demo@example.com",
|
||||
hashed_password="hashed_password"
|
||||
)
|
||||
new_user = await crud_users.create(db=db, object=user_data)
|
||||
print(f"Created user: {new_user.id}")
|
||||
|
||||
# 2. READ
|
||||
user = await crud_users.get(
|
||||
db=db,
|
||||
id=new_user.id,
|
||||
schema_to_select=UserRead
|
||||
)
|
||||
print(f"Retrieved user: {user.username}")
|
||||
|
||||
# 3. UPDATE
|
||||
update_data = UserUpdate(email="updated@example.com")
|
||||
await crud_users.update(db=db, object=update_data, id=new_user.id)
|
||||
print("User updated")
|
||||
|
||||
# 4. DELETE (soft delete)
|
||||
await crud_users.delete(db=db, id=new_user.id)
|
||||
print("User soft deleted")
|
||||
|
||||
# 5. VERIFY DELETION
|
||||
deleted_user = await crud_users.get(db=db, id=new_user.id, is_deleted=True)
|
||||
print(f"User deleted at: {deleted_user.deleted_at}")
|
||||
```
|
||||
|
||||
### Pagination Helper
|
||||
|
||||
Using FastCRUD's pagination utilities:
|
||||
|
||||
```python
|
||||
from fastcrud.paginated import compute_offset, paginated_response
|
||||
|
||||
async def get_paginated_users(
|
||||
db: AsyncSession,
|
||||
page: int = 1,
|
||||
items_per_page: int = 10
|
||||
):
|
||||
users_data = await crud_users.get_multi(
|
||||
db=db,
|
||||
offset=compute_offset(page, items_per_page),
|
||||
limit=items_per_page,
|
||||
is_deleted=False,
|
||||
schema_to_select=UserRead
|
||||
)
|
||||
|
||||
return paginated_response(
|
||||
crud_data=users_data,
|
||||
page=page,
|
||||
items_per_page=items_per_page
|
||||
)
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
|
||||
Proper error handling with CRUD operations:
|
||||
|
||||
```python
|
||||
from app.core.exceptions.http_exceptions import NotFoundException, DuplicateValueException
|
||||
|
||||
async def safe_user_creation(db: AsyncSession, user_data: UserCreate):
|
||||
# Check for duplicates
|
||||
if await crud_users.exists(db=db, email=user_data.email):
|
||||
raise DuplicateValueException("Email already registered")
|
||||
|
||||
if await crud_users.exists(db=db, username=user_data.username):
|
||||
raise DuplicateValueException("Username not available")
|
||||
|
||||
# Create user
|
||||
try:
|
||||
user_internal = UserCreateInternal(**user_data.model_dump())
|
||||
created_user = await crud_users.create(db=db, object=user_internal)
|
||||
return created_user
|
||||
except Exception as e:
|
||||
# Handle database errors
|
||||
await db.rollback()
|
||||
raise e
|
||||
```
|
||||
|
||||
## Performance Tips
|
||||
|
||||
### 1. Use Schema Selection
|
||||
|
||||
Always specify `schema_to_select` to avoid loading unnecessary data:
|
||||
|
||||
```python
|
||||
# Good - only loads needed fields
|
||||
user = await crud_users.get(db=db, id=user_id, schema_to_select=UserRead)
|
||||
|
||||
# Avoid - loads all fields
|
||||
user = await crud_users.get(db=db, id=user_id)
|
||||
```
|
||||
|
||||
### 2. Batch Operations
|
||||
|
||||
For multiple operations, use transactions:
|
||||
|
||||
```python
|
||||
async def batch_user_updates(db: AsyncSession, updates: List[dict]):
|
||||
try:
|
||||
for update in updates:
|
||||
await crud_users.update(db=db, object=update["data"], id=update["id"])
|
||||
await db.commit()
|
||||
except Exception:
|
||||
await db.rollback()
|
||||
raise
|
||||
```
|
||||
|
||||
### 3. Use Exists for Checks
|
||||
|
||||
Use `exists()` instead of `get()` when you only need to check existence:
|
||||
|
||||
```python
|
||||
# Good - faster, doesn't load data
|
||||
if await crud_users.exists(db=db, email=email):
|
||||
raise DuplicateValueException("Email taken")
|
||||
|
||||
# Avoid - slower, loads unnecessary data
|
||||
user = await crud_users.get(db=db, email=email)
|
||||
if user:
|
||||
raise DuplicateValueException("Email taken")
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **[Database Migrations](migrations.md)** - Managing database schema changes
|
||||
- **[API Development](../api/index.md)** - Using CRUD in API endpoints
|
||||
- **[Caching](../caching/index.md)** - Optimizing CRUD with caching
|
||||
235
docs/user-guide/database/index.md
Normal file
235
docs/user-guide/database/index.md
Normal file
@ -0,0 +1,235 @@
|
||||
# Database Layer
|
||||
|
||||
Learn how to work with the database layer in the FastAPI Boilerplate. This section covers everything you need to store and retrieve data effectively.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- **[Models](models.md)** - Define database tables with SQLAlchemy models
|
||||
- **[Schemas](schemas.md)** - Validate and serialize data with Pydantic schemas
|
||||
- **[CRUD Operations](crud.md)** - Perform database operations with FastCRUD
|
||||
- **[Migrations](migrations.md)** - Manage database schema changes with Alembic
|
||||
|
||||
## Quick Overview
|
||||
|
||||
The boilerplate uses a layered architecture that separates concerns:
|
||||
|
||||
```python
|
||||
# API Endpoint
|
||||
@router.post("/", response_model=UserRead)
|
||||
async def create_user(user_data: UserCreate, db: AsyncSession):
|
||||
return await crud_users.create(db=db, object=user_data)
|
||||
|
||||
# The layers work together:
|
||||
# 1. UserCreate schema validates the input
|
||||
# 2. crud_users handles the database operation
|
||||
# 3. User model defines the database table
|
||||
# 4. UserRead schema formats the response
|
||||
```
|
||||
|
||||
## Architecture
|
||||
|
||||
The database layer follows a clear separation:
|
||||
|
||||
```
|
||||
API Request
|
||||
↓
|
||||
Pydantic Schema (validation & serialization)
|
||||
↓
|
||||
CRUD Layer (business logic & database operations)
|
||||
↓
|
||||
SQLAlchemy Model (database table definition)
|
||||
↓
|
||||
PostgreSQL Database
|
||||
```
|
||||
|
||||
## Key Features
|
||||
|
||||
### 🗄️ **SQLAlchemy 2.0 Models**
|
||||
Modern async SQLAlchemy with type hints:
|
||||
```python
|
||||
class User(Base):
|
||||
__tablename__ = "user"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
username: Mapped[str] = mapped_column(String(50), unique=True)
|
||||
email: Mapped[str] = mapped_column(String(100), unique=True)
|
||||
created_at: Mapped[datetime] = mapped_column(default=datetime.utcnow)
|
||||
```
|
||||
|
||||
### ✅ **Pydantic Schemas**
|
||||
Automatic validation and serialization:
|
||||
```python
|
||||
class UserCreate(BaseModel):
|
||||
username: str = Field(min_length=2, max_length=50)
|
||||
email: EmailStr
|
||||
password: str = Field(min_length=8)
|
||||
|
||||
class UserRead(BaseModel):
|
||||
id: int
|
||||
username: str
|
||||
email: str
|
||||
created_at: datetime
|
||||
# Note: no password field in read schema
|
||||
```
|
||||
|
||||
### 🔧 **FastCRUD Operations**
|
||||
Consistent database operations:
|
||||
```python
|
||||
# Create
|
||||
user = await crud_users.create(db=db, object=user_create)
|
||||
|
||||
# Read
|
||||
user = await crud_users.get(db=db, id=user_id)
|
||||
users = await crud_users.get_multi(db=db, offset=0, limit=10)
|
||||
|
||||
# Update
|
||||
user = await crud_users.update(db=db, object=user_update, id=user_id)
|
||||
|
||||
# Delete (soft delete)
|
||||
await crud_users.delete(db=db, id=user_id)
|
||||
```
|
||||
|
||||
### 🔄 **Database Migrations**
|
||||
Track schema changes with Alembic:
|
||||
```bash
|
||||
# Generate migration
|
||||
alembic revision --autogenerate -m "Add user table"
|
||||
|
||||
# Apply migrations
|
||||
alembic upgrade head
|
||||
|
||||
# Rollback if needed
|
||||
alembic downgrade -1
|
||||
```
|
||||
|
||||
## Database Setup
|
||||
|
||||
The boilerplate is configured for PostgreSQL with async support:
|
||||
|
||||
### Environment Configuration
|
||||
```bash
|
||||
# .env file
|
||||
POSTGRES_USER=your_user
|
||||
POSTGRES_PASSWORD=your_password
|
||||
POSTGRES_SERVER=localhost
|
||||
POSTGRES_PORT=5432
|
||||
POSTGRES_DB=your_database
|
||||
```
|
||||
|
||||
### Connection Management
|
||||
```python
|
||||
# Database session dependency
|
||||
async def async_get_db() -> AsyncIterator[AsyncSession]:
|
||||
async with async_session_maker() as session:
|
||||
yield session
|
||||
|
||||
# Use in endpoints
|
||||
@router.get("/users/")
|
||||
async def get_users(db: Annotated[AsyncSession, Depends(async_get_db)]):
|
||||
return await crud_users.get_multi(db=db)
|
||||
```
|
||||
|
||||
## Included Models
|
||||
|
||||
The boilerplate includes four example models:
|
||||
|
||||
### **User Model** - Authentication & user management
|
||||
- Username, email, password (hashed)
|
||||
- Soft delete support
|
||||
- Tier-based access control
|
||||
|
||||
### **Post Model** - Content with user relationships
|
||||
- Title, content, creation metadata
|
||||
- Foreign key to user (no SQLAlchemy relationships)
|
||||
- Soft delete built-in
|
||||
|
||||
### **Tier Model** - User subscription levels
|
||||
- Name-based tiers (free, premium, etc.)
|
||||
- Links to rate limiting system
|
||||
|
||||
### **Rate Limit Model** - API access control
|
||||
- Path-specific rate limits per tier
|
||||
- Configurable limits and time periods
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```text
|
||||
src/app/
|
||||
├── models/ # SQLAlchemy models (database tables)
|
||||
│ ├── __init__.py
|
||||
│ ├── user.py # User table definition
|
||||
│ ├── post.py # Post table definition
|
||||
│ └── ...
|
||||
├── schemas/ # Pydantic schemas (validation)
|
||||
│ ├── __init__.py
|
||||
│ ├── user.py # User validation schemas
|
||||
│ ├── post.py # Post validation schemas
|
||||
│ └── ...
|
||||
├── crud/ # Database operations
|
||||
│ ├── __init__.py
|
||||
│ ├── crud_users.py # User CRUD operations
|
||||
│ ├── crud_posts.py # Post CRUD operations
|
||||
│ └── ...
|
||||
└── core/db/ # Database configuration
|
||||
├── database.py # Connection and session setup
|
||||
└── models.py # Base classes and mixins
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Create with Validation
|
||||
```python
|
||||
@router.post("/users/", response_model=UserRead)
|
||||
async def create_user(
|
||||
user_data: UserCreate, # Validates input automatically
|
||||
db: Annotated[AsyncSession, Depends(async_get_db)]
|
||||
):
|
||||
# Check for duplicates
|
||||
if await crud_users.exists(db=db, email=user_data.email):
|
||||
raise DuplicateValueException("Email already exists")
|
||||
|
||||
# Create user (password gets hashed automatically)
|
||||
return await crud_users.create(db=db, object=user_data)
|
||||
```
|
||||
|
||||
### Query with Filters
|
||||
```python
|
||||
# Get active users only
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
is_active=True,
|
||||
is_deleted=False,
|
||||
offset=0,
|
||||
limit=10
|
||||
)
|
||||
|
||||
# Search users
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
username__icontains="john", # Contains "john"
|
||||
schema_to_select=UserRead
|
||||
)
|
||||
```
|
||||
|
||||
### Soft Delete Pattern
|
||||
```python
|
||||
# Soft delete (sets is_deleted=True)
|
||||
await crud_users.delete(db=db, id=user_id)
|
||||
|
||||
# Hard delete (actually removes from database)
|
||||
await crud_users.db_delete(db=db, id=user_id)
|
||||
|
||||
# Get only non-deleted records
|
||||
users = await crud_users.get_multi(db=db, is_deleted=False)
|
||||
```
|
||||
|
||||
## What's Next
|
||||
|
||||
Each guide builds on the previous one with practical examples:
|
||||
|
||||
1. **[Models](models.md)** - Define your database structure
|
||||
2. **[Schemas](schemas.md)** - Add validation and serialization
|
||||
3. **[CRUD Operations](crud.md)** - Implement business logic
|
||||
4. **[Migrations](migrations.md)** - Deploy changes safely
|
||||
|
||||
The boilerplate provides a solid foundation - just follow these patterns to build your data layer!
|
||||
470
docs/user-guide/database/migrations.md
Normal file
470
docs/user-guide/database/migrations.md
Normal file
@ -0,0 +1,470 @@
|
||||
# Database Migrations
|
||||
|
||||
This guide covers database migrations using Alembic, the migration tool for SQLAlchemy. Learn how to manage database schema changes safely and efficiently in development and production.
|
||||
|
||||
## Overview
|
||||
|
||||
The FastAPI Boilerplate uses [Alembic](https://alembic.sqlalchemy.org/) for database migrations. Alembic provides:
|
||||
|
||||
- **Version-controlled schema changes** - Track every database modification
|
||||
- **Automatic migration generation** - Generate migrations from model changes
|
||||
- **Reversible migrations** - Upgrade and downgrade database versions
|
||||
- **Environment-specific configurations** - Different settings for dev/staging/production
|
||||
- **Safe schema evolution** - Apply changes incrementally
|
||||
|
||||
## Simple Setup: Automatic Table Creation
|
||||
|
||||
For simple projects or development, the boilerplate includes `create_tables_on_start` parameter that automatically creates all tables on application startup:
|
||||
|
||||
```python
|
||||
# This is enabled by default in create_application()
|
||||
app = create_application(
|
||||
router=router,
|
||||
settings=settings,
|
||||
create_tables_on_start=True # Default: True
|
||||
)
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
|
||||
- ✅ **Development** - Quick setup without migration management
|
||||
- ✅ **Simple projects** - When you don't need migration history
|
||||
- ✅ **Prototyping** - Fast iteration without migration complexity
|
||||
- ✅ **Testing** - Clean database state for each test run
|
||||
|
||||
**When NOT to use:**
|
||||
|
||||
- ❌ **Production** - No migration history or rollback capability
|
||||
- ❌ **Team development** - Can't track schema changes between developers
|
||||
- ❌ **Data migrations** - Only handles schema, not data transformations
|
||||
- ❌ **Complex deployments** - No control over when/how schema changes apply
|
||||
|
||||
```python
|
||||
# Disable for production environments
|
||||
app = create_application(
|
||||
router=router,
|
||||
settings=settings,
|
||||
create_tables_on_start=False # Use migrations instead
|
||||
)
|
||||
```
|
||||
|
||||
For production deployments and team development, use proper Alembic migrations as described below.
|
||||
|
||||
## Configuration
|
||||
|
||||
### Alembic Setup
|
||||
|
||||
Alembic is configured in `src/alembic.ini`:
|
||||
|
||||
```ini
|
||||
[alembic]
|
||||
# Path to migration files
|
||||
script_location = migrations
|
||||
|
||||
# Database URL with environment variable substitution
|
||||
sqlalchemy.url = postgresql://%(POSTGRES_USER)s:%(POSTGRES_PASSWORD)s@%(POSTGRES_SERVER)s:%(POSTGRES_PORT)s/%(POSTGRES_DB)s
|
||||
|
||||
# Other configurations
|
||||
file_template = %%(year)d%%(month).2d%%(day).2d_%%(hour).2d%%(minute).2d_%%(rev)s_%%(slug)s
|
||||
timezone = UTC
|
||||
```
|
||||
|
||||
### Environment Configuration
|
||||
|
||||
Migration environment is configured in `src/migrations/env.py`:
|
||||
|
||||
```python
|
||||
# src/migrations/env.py
|
||||
from alembic import context
|
||||
from sqlalchemy import engine_from_config, pool
|
||||
from app.core.db.database import Base
|
||||
from app.core.config import settings
|
||||
|
||||
# Import all models to ensure they're registered
|
||||
from app.models import * # This imports all models
|
||||
|
||||
config = context.config
|
||||
|
||||
# Override database URL from environment
|
||||
config.set_main_option("sqlalchemy.url", settings.DATABASE_URL)
|
||||
|
||||
target_metadata = Base.metadata
|
||||
```
|
||||
|
||||
## Migration Workflow
|
||||
|
||||
### 1. Creating Migrations
|
||||
|
||||
Generate migrations automatically when you change models:
|
||||
|
||||
```bash
|
||||
# Navigate to src directory
|
||||
cd src
|
||||
|
||||
# Generate migration from model changes
|
||||
uv run alembic revision --autogenerate -m "Add user profile fields"
|
||||
```
|
||||
|
||||
**What happens:**
|
||||
- Alembic compares current models with database schema
|
||||
- Generates a new migration file in `src/migrations/versions/`
|
||||
- Migration includes upgrade and downgrade functions
|
||||
|
||||
### 2. Review Generated Migration
|
||||
|
||||
Always review auto-generated migrations before applying:
|
||||
|
||||
```python
|
||||
# Example migration file: src/migrations/versions/20241215_1430_add_user_profile_fields.py
|
||||
"""Add user profile fields
|
||||
|
||||
Revision ID: abc123def456
|
||||
Revises: previous_revision_id
|
||||
Create Date: 2024-12-15 14:30:00.000000
|
||||
|
||||
"""
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
# revision identifiers
|
||||
revision = 'abc123def456'
|
||||
down_revision = 'previous_revision_id'
|
||||
branch_labels = None
|
||||
depends_on = None
|
||||
|
||||
def upgrade() -> None:
|
||||
# Add new columns
|
||||
op.add_column('user', sa.Column('bio', sa.String(500), nullable=True))
|
||||
op.add_column('user', sa.Column('website', sa.String(255), nullable=True))
|
||||
|
||||
# Create index
|
||||
op.create_index('ix_user_website', 'user', ['website'])
|
||||
|
||||
def downgrade() -> None:
|
||||
# Remove changes (reverse order)
|
||||
op.drop_index('ix_user_website', 'user')
|
||||
op.drop_column('user', 'website')
|
||||
op.drop_column('user', 'bio')
|
||||
```
|
||||
|
||||
### 3. Apply Migration
|
||||
|
||||
Apply migrations to update database schema:
|
||||
|
||||
```bash
|
||||
# Apply all pending migrations
|
||||
uv run alembic upgrade head
|
||||
|
||||
# Apply specific number of migrations
|
||||
uv run alembic upgrade +2
|
||||
|
||||
# Apply to specific revision
|
||||
uv run alembic upgrade abc123def456
|
||||
```
|
||||
|
||||
### 4. Verify Migration
|
||||
|
||||
Check migration status and current version:
|
||||
|
||||
```bash
|
||||
# Show current database version
|
||||
uv run alembic current
|
||||
|
||||
# Show migration history
|
||||
uv run alembic history
|
||||
|
||||
# Show pending migrations
|
||||
uv run alembic show head
|
||||
```
|
||||
|
||||
## Common Migration Scenarios
|
||||
|
||||
### Adding New Model
|
||||
|
||||
1. **Create the model** in `src/app/models/`:
|
||||
|
||||
```python
|
||||
# src/app/models/category.py
|
||||
from sqlalchemy import String, DateTime
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
from datetime import datetime
|
||||
from app.core.db.database import Base
|
||||
|
||||
class Category(Base):
|
||||
__tablename__ = "category"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True, init=False)
|
||||
name: Mapped[str] = mapped_column(String(50), unique=True, nullable=False)
|
||||
slug: Mapped[str] = mapped_column(String(50), unique=True, nullable=False)
|
||||
description: Mapped[str] = mapped_column(String(255), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow)
|
||||
```
|
||||
|
||||
2. **Import in __init__.py**:
|
||||
|
||||
```python
|
||||
# src/app/models/__init__.py
|
||||
from .user import User
|
||||
from .post import Post
|
||||
from .tier import Tier
|
||||
from .rate_limit import RateLimit
|
||||
from .category import Category # Add new import
|
||||
```
|
||||
|
||||
3. **Generate migration**:
|
||||
|
||||
```bash
|
||||
uv run alembic revision --autogenerate -m "Add category model"
|
||||
```
|
||||
|
||||
### Adding Foreign Key
|
||||
|
||||
1. **Update model with foreign key**:
|
||||
|
||||
```python
|
||||
# Add to Post model
|
||||
category_id: Mapped[Optional[int]] = mapped_column(ForeignKey("category.id"), nullable=True)
|
||||
```
|
||||
|
||||
2. **Generate migration**:
|
||||
|
||||
```bash
|
||||
uv run alembic revision --autogenerate -m "Add category_id to posts"
|
||||
```
|
||||
|
||||
3. **Review and apply**:
|
||||
|
||||
```python
|
||||
# Generated migration will include:
|
||||
def upgrade() -> None:
|
||||
op.add_column('post', sa.Column('category_id', sa.Integer(), nullable=True))
|
||||
op.create_foreign_key('fk_post_category_id', 'post', 'category', ['category_id'], ['id'])
|
||||
op.create_index('ix_post_category_id', 'post', ['category_id'])
|
||||
```
|
||||
|
||||
### Data Migrations
|
||||
|
||||
Sometimes you need to migrate data, not just schema:
|
||||
|
||||
```python
|
||||
# Example: Populate default category for existing posts
|
||||
def upgrade() -> None:
|
||||
# Add the column
|
||||
op.add_column('post', sa.Column('category_id', sa.Integer(), nullable=True))
|
||||
|
||||
# Data migration
|
||||
connection = op.get_bind()
|
||||
|
||||
# Create default category
|
||||
connection.execute(
|
||||
"INSERT INTO category (name, slug, description) VALUES ('General', 'general', 'Default category')"
|
||||
)
|
||||
|
||||
# Get default category ID
|
||||
result = connection.execute("SELECT id FROM category WHERE slug = 'general'")
|
||||
default_category_id = result.fetchone()[0]
|
||||
|
||||
# Update existing posts
|
||||
connection.execute(
|
||||
f"UPDATE post SET category_id = {default_category_id} WHERE category_id IS NULL"
|
||||
)
|
||||
|
||||
# Make column non-nullable after data migration
|
||||
op.alter_column('post', 'category_id', nullable=False)
|
||||
```
|
||||
|
||||
### Renaming Columns
|
||||
|
||||
```python
|
||||
def upgrade() -> None:
|
||||
# Rename column
|
||||
op.alter_column('user', 'full_name', new_column_name='name')
|
||||
|
||||
def downgrade() -> None:
|
||||
# Reverse the rename
|
||||
op.alter_column('user', 'name', new_column_name='full_name')
|
||||
```
|
||||
|
||||
### Dropping Tables
|
||||
|
||||
```python
|
||||
def upgrade() -> None:
|
||||
# Drop table (be careful!)
|
||||
op.drop_table('old_table')
|
||||
|
||||
def downgrade() -> None:
|
||||
# Recreate table structure
|
||||
op.create_table('old_table',
|
||||
sa.Column('id', sa.Integer(), nullable=False),
|
||||
sa.Column('name', sa.String(50), nullable=True),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
```
|
||||
|
||||
## Production Migration Strategy
|
||||
|
||||
### 1. Development Workflow
|
||||
|
||||
```bash
|
||||
# 1. Make model changes
|
||||
# 2. Generate migration
|
||||
uv run alembic revision --autogenerate -m "Descriptive message"
|
||||
|
||||
# 3. Review migration file
|
||||
# 4. Test migration
|
||||
uv run alembic upgrade head
|
||||
|
||||
# 5. Test downgrade (optional)
|
||||
uv run alembic downgrade -1
|
||||
uv run alembic upgrade head
|
||||
```
|
||||
|
||||
### 2. Staging Deployment
|
||||
|
||||
```bash
|
||||
# 1. Deploy code with migrations
|
||||
# 2. Backup database
|
||||
pg_dump -h staging-db -U user dbname > backup_$(date +%Y%m%d_%H%M%S).sql
|
||||
|
||||
# 3. Apply migrations
|
||||
uv run alembic upgrade head
|
||||
|
||||
# 4. Verify application works
|
||||
# 5. Run tests
|
||||
```
|
||||
|
||||
### 3. Production Deployment
|
||||
|
||||
```bash
|
||||
# 1. Schedule maintenance window
|
||||
# 2. Create database backup
|
||||
pg_dump -h prod-db -U user dbname > prod_backup_$(date +%Y%m%d_%H%M%S).sql
|
||||
|
||||
# 3. Apply migrations (with monitoring)
|
||||
uv run alembic upgrade head
|
||||
|
||||
# 4. Verify health checks pass
|
||||
# 5. Monitor application metrics
|
||||
```
|
||||
|
||||
## Docker Considerations
|
||||
|
||||
### Development with Docker Compose
|
||||
|
||||
For local development, migrations run automatically:
|
||||
|
||||
```yaml
|
||||
# docker-compose.yml
|
||||
services:
|
||||
web:
|
||||
# ... other config
|
||||
depends_on:
|
||||
- db
|
||||
command: |
|
||||
sh -c "
|
||||
uv run alembic upgrade head &&
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
"
|
||||
```
|
||||
|
||||
### Production Docker
|
||||
|
||||
In production, run migrations separately:
|
||||
|
||||
```dockerfile
|
||||
# Dockerfile migration stage
|
||||
FROM python:3.11-slim as migration
|
||||
COPY requirements.txt .
|
||||
RUN pip install -r requirements.txt
|
||||
COPY src/ /app/
|
||||
WORKDIR /app
|
||||
CMD ["alembic", "upgrade", "head"]
|
||||
```
|
||||
|
||||
```yaml
|
||||
# docker-compose.prod.yml
|
||||
services:
|
||||
migrate:
|
||||
build:
|
||||
context: .
|
||||
target: migration
|
||||
env_file:
|
||||
- .env
|
||||
depends_on:
|
||||
- db
|
||||
command: alembic upgrade head
|
||||
|
||||
web:
|
||||
# ... web service config
|
||||
depends_on:
|
||||
- migrate
|
||||
```
|
||||
|
||||
## Migration Best Practices
|
||||
|
||||
### 1. Always Review Generated Migrations
|
||||
|
||||
```python
|
||||
# Check for issues like:
|
||||
# - Missing imports
|
||||
# - Incorrect nullable settings
|
||||
# - Missing indexes
|
||||
# - Data loss operations
|
||||
```
|
||||
|
||||
### 2. Use Descriptive Messages
|
||||
|
||||
```bash
|
||||
# Good
|
||||
uv run alembic revision --autogenerate -m "Add user email verification fields"
|
||||
|
||||
# Bad
|
||||
uv run alembic revision --autogenerate -m "Update user model"
|
||||
```
|
||||
|
||||
### 3. Handle Nullable Columns Carefully
|
||||
|
||||
```python
|
||||
# When adding non-nullable columns to existing tables:
|
||||
def upgrade() -> None:
|
||||
# 1. Add as nullable first
|
||||
op.add_column('user', sa.Column('phone', sa.String(20), nullable=True))
|
||||
|
||||
# 2. Populate with default data
|
||||
op.execute("UPDATE user SET phone = '' WHERE phone IS NULL")
|
||||
|
||||
# 3. Make non-nullable
|
||||
op.alter_column('user', 'phone', nullable=False)
|
||||
```
|
||||
|
||||
### 4. Test Rollbacks
|
||||
|
||||
```bash
|
||||
# Test that your downgrade works
|
||||
uv run alembic downgrade -1
|
||||
uv run alembic upgrade head
|
||||
```
|
||||
|
||||
### 5. Use Transactions for Complex Migrations
|
||||
|
||||
```python
|
||||
def upgrade() -> None:
|
||||
# Complex migration with transaction
|
||||
connection = op.get_bind()
|
||||
trans = connection.begin()
|
||||
try:
|
||||
# Multiple operations
|
||||
op.create_table(...)
|
||||
op.add_column(...)
|
||||
connection.execute("UPDATE ...")
|
||||
trans.commit()
|
||||
except:
|
||||
trans.rollback()
|
||||
raise
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **[CRUD Operations](crud.md)** - Working with migrated database schema
|
||||
- **[API Development](../api/index.md)** - Building endpoints for your models
|
||||
- **[Testing](../testing.md)** - Testing database migrations
|
||||
484
docs/user-guide/database/models.md
Normal file
484
docs/user-guide/database/models.md
Normal file
@ -0,0 +1,484 @@
|
||||
# Database Models
|
||||
|
||||
This section explains how SQLAlchemy models are implemented in the boilerplate, how to create new models, and the patterns used for relationships, validation, and data integrity.
|
||||
|
||||
## Model Structure
|
||||
|
||||
Models are defined in `src/app/models/` using SQLAlchemy 2.0's declarative syntax with `Mapped` type annotations.
|
||||
|
||||
### Base Model
|
||||
|
||||
All models inherit from `Base` defined in `src/app/core/db/database.py`:
|
||||
|
||||
```python
|
||||
from sqlalchemy.orm import DeclarativeBase
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
pass
|
||||
```
|
||||
|
||||
**SQLAlchemy 2.0 Change**: Uses `DeclarativeBase` instead of the older `declarative_base()` function. This provides better type checking and IDE support.
|
||||
|
||||
### Model File Structure
|
||||
|
||||
Each model is in its own file:
|
||||
|
||||
```text
|
||||
src/app/models/
|
||||
├── __init__.py # Imports all models for Alembic discovery
|
||||
├── user.py # User authentication model
|
||||
├── post.py # Example content model with relationships
|
||||
├── tier.py # User subscription tiers
|
||||
└── rate_limit.py # API rate limiting configuration
|
||||
```
|
||||
|
||||
**Import Requirement**: Models must be imported in `__init__.py` for Alembic to detect them during migration generation.
|
||||
|
||||
## Design Decision: No SQLAlchemy Relationships
|
||||
|
||||
The boilerplate deliberately avoids using SQLAlchemy's `relationship()` feature. This is an intentional architectural choice with specific benefits.
|
||||
|
||||
### Why No Relationships
|
||||
|
||||
**Performance Concerns**:
|
||||
|
||||
- **N+1 Query Problem**: Relationships can trigger multiple queries when accessing related data
|
||||
- **Lazy Loading**: Unpredictable when queries execute, making performance optimization difficult
|
||||
- **Memory Usage**: Loading large object graphs consumes significant memory
|
||||
|
||||
**Code Clarity**:
|
||||
|
||||
- **Explicit Data Fetching**: Developers see exactly what data is being loaded and when
|
||||
- **Predictable Queries**: No "magic" queries triggered by attribute access
|
||||
- **Easier Debugging**: SQL queries are explicit in the code, not hidden in relationship configuration
|
||||
|
||||
**Flexibility**:
|
||||
|
||||
- **Query Optimization**: Can optimize each query for its specific use case
|
||||
- **Selective Loading**: Load only the fields needed for each operation
|
||||
- **Join Control**: Use FastCRUD's join methods when needed, skip when not
|
||||
|
||||
### What This Means in Practice
|
||||
|
||||
Instead of this (traditional SQLAlchemy):
|
||||
```python
|
||||
# Not used in the boilerplate
|
||||
class User(Base):
|
||||
posts: Mapped[List["Post"]] = relationship("Post", back_populates="created_by_user")
|
||||
|
||||
class Post(Base):
|
||||
created_by_user: Mapped["User"] = relationship("User", back_populates="posts")
|
||||
```
|
||||
|
||||
The boilerplate uses this approach:
|
||||
```python
|
||||
# DO - Explicit and controlled
|
||||
class User(Base):
|
||||
# Only foreign key, no relationship
|
||||
tier_id: Mapped[int | None] = mapped_column(ForeignKey("tier.id"), index=True, default=None)
|
||||
|
||||
class Post(Base):
|
||||
# Only foreign key, no relationship
|
||||
created_by_user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), index=True)
|
||||
|
||||
# Explicit queries - you control exactly what's loaded
|
||||
user = await crud_users.get(db=db, id=1)
|
||||
posts = await crud_posts.get_multi(db=db, created_by_user_id=user.id)
|
||||
|
||||
# Or use joins when needed
|
||||
posts_with_users = await crud_posts.get_multi_joined(
|
||||
db=db,
|
||||
join_model=User,
|
||||
schema_to_select=PostRead,
|
||||
join_schema_to_select=UserRead
|
||||
)
|
||||
```
|
||||
|
||||
### Benefits of This Approach
|
||||
|
||||
**Predictable Performance**:
|
||||
|
||||
- Every database query is explicit in the code
|
||||
- No surprise queries from accessing relationships
|
||||
- Easier to identify and optimize slow operations
|
||||
|
||||
**Better Caching**:
|
||||
|
||||
- Can cache individual models without worrying about related data
|
||||
- Cache invalidation is simpler and more predictable
|
||||
|
||||
**API Design**:
|
||||
|
||||
- Forces thinking about what data clients actually need
|
||||
- Prevents over-fetching in API responses
|
||||
- Encourages lean, focused endpoints
|
||||
|
||||
**Testing**:
|
||||
|
||||
- Easier to mock database operations
|
||||
- No complex relationship setup in test fixtures
|
||||
- More predictable test data requirements
|
||||
|
||||
### When You Need Related Data
|
||||
|
||||
Use FastCRUD's join capabilities:
|
||||
|
||||
```python
|
||||
# Single record with related data
|
||||
post_with_author = await crud_posts.get_joined(
|
||||
db=db,
|
||||
join_model=User,
|
||||
schema_to_select=PostRead,
|
||||
join_schema_to_select=UserRead,
|
||||
id=post_id
|
||||
)
|
||||
|
||||
# Multiple records with joins
|
||||
posts_with_authors = await crud_posts.get_multi_joined(
|
||||
db=db,
|
||||
join_model=User,
|
||||
offset=0,
|
||||
limit=10
|
||||
)
|
||||
```
|
||||
|
||||
### Alternative Approaches
|
||||
|
||||
If you need relationships in your project, you can add them:
|
||||
|
||||
```python
|
||||
# Add relationships if needed for your use case
|
||||
from sqlalchemy.orm import relationship
|
||||
|
||||
class User(Base):
|
||||
# ... existing fields ...
|
||||
posts: Mapped[List["Post"]] = relationship("Post", back_populates="created_by_user")
|
||||
|
||||
class Post(Base):
|
||||
# ... existing fields ...
|
||||
created_by_user: Mapped["User"] = relationship("User", back_populates="posts")
|
||||
```
|
||||
|
||||
But consider the trade-offs and whether explicit queries might be better for your use case.
|
||||
|
||||
## User Model Implementation
|
||||
|
||||
The User model (`src/app/models/user.py`) demonstrates authentication patterns:
|
||||
|
||||
```python
|
||||
import uuid as uuid_pkg
|
||||
from datetime import UTC, datetime
|
||||
from sqlalchemy import DateTime, ForeignKey, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
from ..core.db.database import Base
|
||||
|
||||
class User(Base):
|
||||
__tablename__ = "user"
|
||||
|
||||
id: Mapped[int] = mapped_column("id", autoincrement=True, nullable=False, unique=True, primary_key=True, init=False)
|
||||
|
||||
# User data
|
||||
name: Mapped[str] = mapped_column(String(30))
|
||||
username: Mapped[str] = mapped_column(String(20), unique=True, index=True)
|
||||
email: Mapped[str] = mapped_column(String(50), unique=True, index=True)
|
||||
hashed_password: Mapped[str] = mapped_column(String)
|
||||
|
||||
# Profile
|
||||
profile_image_url: Mapped[str] = mapped_column(String, default="https://profileimageurl.com")
|
||||
|
||||
# UUID for external references
|
||||
uuid: Mapped[uuid_pkg.UUID] = mapped_column(default_factory=uuid_pkg.uuid4, primary_key=True, unique=True)
|
||||
|
||||
# Timestamps
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default_factory=lambda: datetime.now(UTC))
|
||||
updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None)
|
||||
deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None)
|
||||
|
||||
# Status flags
|
||||
is_deleted: Mapped[bool] = mapped_column(default=False, index=True)
|
||||
is_superuser: Mapped[bool] = mapped_column(default=False)
|
||||
|
||||
# Foreign key to tier system (no relationship defined)
|
||||
tier_id: Mapped[int | None] = mapped_column(ForeignKey("tier.id"), index=True, default=None, init=False)
|
||||
```
|
||||
|
||||
### Key Implementation Details
|
||||
|
||||
**Type Annotations**: `Mapped[type]` provides type hints for SQLAlchemy 2.0. IDE and mypy can validate types.
|
||||
|
||||
**String Lengths**: Explicit lengths (`String(50)`) prevent database errors and define constraints clearly.
|
||||
|
||||
**Nullable Fields**: Explicitly set `nullable=False` for required fields, `nullable=True` for optional ones.
|
||||
|
||||
**Default Values**: Use `default=` for database-level defaults, Python functions for computed defaults.
|
||||
|
||||
## Post Model with Relationships
|
||||
|
||||
The Post model (`src/app/models/post.py`) shows relationships and soft deletion:
|
||||
|
||||
```python
|
||||
import uuid as uuid_pkg
|
||||
from datetime import UTC, datetime
|
||||
from sqlalchemy import DateTime, ForeignKey, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
from ..core.db.database import Base
|
||||
|
||||
class Post(Base):
|
||||
__tablename__ = "post"
|
||||
|
||||
id: Mapped[int] = mapped_column("id", autoincrement=True, nullable=False, unique=True, primary_key=True, init=False)
|
||||
|
||||
# Content
|
||||
title: Mapped[str] = mapped_column(String(30))
|
||||
text: Mapped[str] = mapped_column(String(63206)) # Large text field
|
||||
media_url: Mapped[str | None] = mapped_column(String, default=None)
|
||||
|
||||
# UUID for external references
|
||||
uuid: Mapped[uuid_pkg.UUID] = mapped_column(default_factory=uuid_pkg.uuid4, primary_key=True, unique=True)
|
||||
|
||||
# Foreign key (no relationship defined)
|
||||
created_by_user_id: Mapped[int] = mapped_column(ForeignKey("user.id"), index=True)
|
||||
|
||||
# Timestamps (built-in soft delete pattern)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default_factory=lambda: datetime.now(UTC))
|
||||
updated_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None)
|
||||
deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None)
|
||||
is_deleted: Mapped[bool] = mapped_column(default=False, index=True)
|
||||
```
|
||||
|
||||
### Soft Deletion Pattern
|
||||
|
||||
Soft deletion is built directly into models:
|
||||
|
||||
```python
|
||||
# Built into each model that needs soft deletes
|
||||
class Post(Base):
|
||||
# ... other fields ...
|
||||
|
||||
# Soft delete fields
|
||||
is_deleted: Mapped[bool] = mapped_column(default=False, index=True)
|
||||
deleted_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), default=None)
|
||||
```
|
||||
|
||||
**Usage**: When `crud_posts.delete()` is called, it sets `is_deleted=True` and `deleted_at=datetime.now(UTC)` instead of removing the database row.
|
||||
|
||||
## Tier and Rate Limiting Models
|
||||
|
||||
### Tier Model
|
||||
|
||||
```python
|
||||
# src/app/models/tier.py
|
||||
class Tier(Base):
|
||||
__tablename__ = "tier"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True, init=False)
|
||||
name: Mapped[str] = mapped_column(String(50), unique=True, nullable=False)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
||||
```
|
||||
|
||||
### Rate Limit Model
|
||||
|
||||
```python
|
||||
# src/app/models/rate_limit.py
|
||||
class RateLimit(Base):
|
||||
__tablename__ = "rate_limit"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True, init=False)
|
||||
tier_id: Mapped[int] = mapped_column(ForeignKey("tier.id"), nullable=False)
|
||||
path: Mapped[str] = mapped_column(String(255), nullable=False)
|
||||
limit: Mapped[int] = mapped_column(nullable=False) # requests allowed
|
||||
period: Mapped[int] = mapped_column(nullable=False) # time period in seconds
|
||||
name: Mapped[Optional[str]] = mapped_column(String(100), nullable=True)
|
||||
```
|
||||
|
||||
**Purpose**: Links API endpoints (`path`) to rate limits (`limit` requests per `period` seconds) for specific user tiers.
|
||||
|
||||
## Creating New Models
|
||||
|
||||
### Step-by-Step Process
|
||||
|
||||
1. **Create model file** in `src/app/models/your_model.py`
|
||||
2. **Define model class** inheriting from `Base`
|
||||
3. **Add to imports** in `src/app/models/__init__.py`
|
||||
4. **Generate migration** with `alembic revision --autogenerate`
|
||||
5. **Apply migration** with `alembic upgrade head`
|
||||
|
||||
### Example: Creating a Category Model
|
||||
|
||||
```python
|
||||
# src/app/models/category.py
|
||||
from datetime import datetime
|
||||
from typing import List
|
||||
from sqlalchemy import String, DateTime
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
from app.core.db.database import Base
|
||||
|
||||
class Category(Base):
|
||||
__tablename__ = "category"
|
||||
|
||||
id: Mapped[int] = mapped_column(primary_key=True, autoincrement=True, init=False)
|
||||
name: Mapped[str] = mapped_column(String(50), unique=True, nullable=False)
|
||||
description: Mapped[str] = mapped_column(String(255), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
||||
```
|
||||
|
||||
If you want to relate Category to Post, just add the id reference in the model:
|
||||
|
||||
```python
|
||||
class Post(Base):
|
||||
__tablename__ = "post"
|
||||
...
|
||||
|
||||
# Foreign key (no relationship defined)
|
||||
category_id: Mapped[int] = mapped_column(ForeignKey("category.id"), index=True)
|
||||
```
|
||||
|
||||
### Import in __init__.py
|
||||
|
||||
```python
|
||||
# src/app/models/__init__.py
|
||||
from .user import User
|
||||
from .post import Post
|
||||
from .tier import Tier
|
||||
from .rate_limit import RateLimit
|
||||
from .category import Category # Add new model
|
||||
```
|
||||
|
||||
**Critical**: Without this import, Alembic won't detect the model for migrations.
|
||||
|
||||
## Model Validation and Constraints
|
||||
|
||||
### Database-Level Constraints
|
||||
|
||||
```python
|
||||
from sqlalchemy import CheckConstraint, Index
|
||||
|
||||
class Product(Base):
|
||||
__tablename__ = "product"
|
||||
|
||||
price: Mapped[float] = mapped_column(nullable=False)
|
||||
quantity: Mapped[int] = mapped_column(nullable=False)
|
||||
|
||||
# Table-level constraints
|
||||
__table_args__ = (
|
||||
CheckConstraint('price > 0', name='positive_price'),
|
||||
CheckConstraint('quantity >= 0', name='non_negative_quantity'),
|
||||
Index('idx_product_price', 'price'),
|
||||
)
|
||||
```
|
||||
|
||||
### Unique Constraints
|
||||
|
||||
```python
|
||||
# Single column unique
|
||||
email: Mapped[str] = mapped_column(String(100), unique=True)
|
||||
|
||||
# Multi-column unique constraint
|
||||
__table_args__ = (
|
||||
UniqueConstraint('user_id', 'category_id', name='unique_user_category'),
|
||||
)
|
||||
```
|
||||
|
||||
## Common Model Patterns
|
||||
|
||||
### Timestamp Tracking
|
||||
|
||||
```python
|
||||
class TimestampedModel:
|
||||
created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, nullable=False)
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime,
|
||||
default=datetime.utcnow,
|
||||
onupdate=datetime.utcnow,
|
||||
nullable=False
|
||||
)
|
||||
|
||||
# Use as mixin
|
||||
class Post(Base, TimestampedModel, SoftDeleteMixin):
|
||||
# Model automatically gets created_at, updated_at, is_deleted, deleted_at
|
||||
__tablename__ = "post"
|
||||
id: Mapped[int] = mapped_column(primary_key=True)
|
||||
```
|
||||
|
||||
### Enumeration Fields
|
||||
|
||||
```python
|
||||
from enum import Enum
|
||||
from sqlalchemy import Enum as SQLEnum
|
||||
|
||||
class UserStatus(Enum):
|
||||
ACTIVE = "active"
|
||||
INACTIVE = "inactive"
|
||||
SUSPENDED = "suspended"
|
||||
|
||||
class User(Base):
|
||||
status: Mapped[UserStatus] = mapped_column(SQLEnum(UserStatus), default=UserStatus.ACTIVE)
|
||||
```
|
||||
|
||||
### JSON Fields
|
||||
|
||||
```python
|
||||
from sqlalchemy.dialects.postgresql import JSONB
|
||||
|
||||
class UserProfile(Base):
|
||||
preferences: Mapped[dict] = mapped_column(JSONB, nullable=True)
|
||||
metadata: Mapped[dict] = mapped_column(JSONB, default=lambda: {})
|
||||
```
|
||||
|
||||
**PostgreSQL-specific**: Uses JSONB for efficient JSON storage and querying.
|
||||
|
||||
## Model Testing
|
||||
|
||||
### Basic Model Tests
|
||||
|
||||
```python
|
||||
# tests/test_models.py
|
||||
import pytest
|
||||
from sqlalchemy.exc import IntegrityError
|
||||
from app.models.user import User
|
||||
|
||||
def test_user_creation():
|
||||
user = User(
|
||||
username="testuser",
|
||||
email="test@example.com",
|
||||
hashed_password="hashed123"
|
||||
)
|
||||
assert user.username == "testuser"
|
||||
assert user.is_active is True # Default value
|
||||
|
||||
def test_user_unique_constraint():
|
||||
# Test that duplicate emails raise IntegrityError
|
||||
with pytest.raises(IntegrityError):
|
||||
# Create users with same email
|
||||
pass
|
||||
```
|
||||
|
||||
## Migration Considerations
|
||||
|
||||
### Backwards Compatible Changes
|
||||
|
||||
Safe changes that don't break existing code:
|
||||
|
||||
- Adding nullable columns
|
||||
- Adding new tables
|
||||
- Adding indexes
|
||||
- Increasing column lengths
|
||||
|
||||
### Breaking Changes
|
||||
|
||||
Changes requiring careful migration:
|
||||
|
||||
- Making columns non-nullable
|
||||
- Removing columns
|
||||
- Changing column types
|
||||
- Removing tables
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that you understand model implementation:
|
||||
|
||||
1. **[Schemas](schemas.md)** - Learn Pydantic validation and serialization
|
||||
2. **[CRUD Operations](crud.md)** - Implement database operations with FastCRUD
|
||||
3. **[Migrations](migrations.md)** - Manage schema changes with Alembic
|
||||
|
||||
The next section covers how Pydantic schemas provide validation and API contracts separate from database models.
|
||||
650
docs/user-guide/database/schemas.md
Normal file
650
docs/user-guide/database/schemas.md
Normal file
@ -0,0 +1,650 @@
|
||||
# Database Schemas
|
||||
|
||||
This section explains how Pydantic schemas handle data validation, serialization, and API contracts in the boilerplate. Schemas are separate from SQLAlchemy models and define what data enters and exits your API.
|
||||
|
||||
## Schema Purpose and Structure
|
||||
|
||||
Schemas serve three main purposes:
|
||||
|
||||
1. **Input Validation** - Validate incoming API request data
|
||||
2. **Output Serialization** - Format database data for API responses
|
||||
3. **API Contracts** - Define clear interfaces between frontend and backend
|
||||
|
||||
### Schema File Organization
|
||||
|
||||
Schemas are organized in `src/app/schemas/` with one file per model:
|
||||
|
||||
```text
|
||||
src/app/schemas/
|
||||
├── __init__.py # Imports for easy access
|
||||
├── user.py # User-related schemas
|
||||
├── post.py # Post-related schemas
|
||||
├── tier.py # Tier schemas
|
||||
├── rate_limit.py # Rate limit schemas
|
||||
└── job.py # Background job schemas
|
||||
```
|
||||
|
||||
## User Schema Implementation
|
||||
|
||||
The User schemas (`src/app/schemas/user.py`) demonstrate common validation patterns:
|
||||
|
||||
```python
|
||||
from datetime import datetime
|
||||
from typing import Annotated
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, EmailStr, Field
|
||||
|
||||
from ..core.schemas import PersistentDeletion, TimestampSchema, UUIDSchema
|
||||
|
||||
|
||||
# Base schema with common fields
|
||||
class UserBase(BaseModel):
|
||||
name: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=2,
|
||||
max_length=30,
|
||||
examples=["User Userson"]
|
||||
)
|
||||
]
|
||||
username: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=2,
|
||||
max_length=20,
|
||||
pattern=r"^[a-z0-9]+$",
|
||||
examples=["userson"]
|
||||
)
|
||||
]
|
||||
email: Annotated[EmailStr, Field(examples=["user.userson@example.com"])]
|
||||
|
||||
|
||||
# Full User data
|
||||
class User(TimestampSchema, UserBase, UUIDSchema, PersistentDeletion):
|
||||
profile_image_url: Annotated[
|
||||
str,
|
||||
Field(default="https://www.profileimageurl.com")
|
||||
]
|
||||
hashed_password: str
|
||||
is_superuser: bool = False
|
||||
tier_id: int | None = None
|
||||
|
||||
|
||||
# Schema for reading user data (API output)
|
||||
class UserRead(BaseModel):
|
||||
id: int
|
||||
|
||||
name: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=2,
|
||||
max_length=30,
|
||||
examples=["User Userson"]
|
||||
)
|
||||
]
|
||||
username: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=2,
|
||||
max_length=20,
|
||||
pattern=r"^[a-z0-9]+$",
|
||||
examples=["userson"]
|
||||
)
|
||||
]
|
||||
email: Annotated[EmailStr, Field(examples=["user.userson@example.com"])]
|
||||
profile_image_url: str
|
||||
tier_id: int | None
|
||||
|
||||
|
||||
# Schema for creating new users (API input)
|
||||
class UserCreate(UserBase): # Inherits from UserBase
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
password: Annotated[
|
||||
str,
|
||||
Field(
|
||||
pattern=r"^.{8,}|[0-9]+|[A-Z]+|[a-z]+|[^a-zA-Z0-9]+$",
|
||||
examples=["Str1ngst!"]
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
# Schema that FastCRUD will use to store just the hash
|
||||
class UserCreateInternal(UserBase):
|
||||
hashed_password: str
|
||||
|
||||
|
||||
# Schema for updating users
|
||||
class UserUpdate(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
name: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
min_length=2,
|
||||
max_length=30,
|
||||
examples=["User Userberg"],
|
||||
default=None
|
||||
)
|
||||
]
|
||||
username: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
min_length=2,
|
||||
max_length=20,
|
||||
pattern=r"^[a-z0-9]+$",
|
||||
examples=["userberg"],
|
||||
default=None
|
||||
)
|
||||
]
|
||||
email: Annotated[
|
||||
EmailStr | None,
|
||||
Field(
|
||||
examples=["user.userberg@example.com"],
|
||||
default=None
|
||||
)
|
||||
]
|
||||
profile_image_url: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
pattern=r"^(https?|ftp)://[^\s/$.?#].[^\s]*$",
|
||||
examples=["https://www.profileimageurl.com"],
|
||||
default=None
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
# Internal update schema
|
||||
class UserUpdateInternal(UserUpdate):
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
# Schema to update tier id
|
||||
class UserTierUpdate(BaseModel):
|
||||
tier_id: int
|
||||
|
||||
|
||||
# Schema for user deletion (soft delete timestamps)
|
||||
class UserDelete(BaseModel):
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
is_deleted: bool
|
||||
deleted_at: datetime
|
||||
|
||||
|
||||
# User specific schema
|
||||
class UserRestoreDeleted(BaseModel):
|
||||
is_deleted: bool
|
||||
```
|
||||
|
||||
### Key Implementation Details
|
||||
|
||||
**Field Validation**: Uses `Annotated[type, Field(...)]` for validation rules. `Field` parameters include:
|
||||
|
||||
- `min_length/max_length` - String length constraints
|
||||
- `gt/ge/lt/le` - Numeric constraints
|
||||
- `pattern` - Pattern matching (regex)
|
||||
- `default` - Default values
|
||||
|
||||
**EmailStr**: Validates email format and normalizes the value.
|
||||
|
||||
**ConfigDict**: Replaces the old `Config` class. `from_attributes=True` allows creating schemas from SQLAlchemy model instances.
|
||||
|
||||
**Internal vs External**: Separate schemas for internal operations (like password hashing) vs API exposure.
|
||||
|
||||
## Schema Patterns
|
||||
|
||||
### Base Schema Pattern
|
||||
|
||||
```python
|
||||
# Common fields shared across operations
|
||||
class PostBase(BaseModel):
|
||||
title: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=1,
|
||||
max_length=100
|
||||
)
|
||||
]
|
||||
content: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=1,
|
||||
max_length=10000
|
||||
)
|
||||
]
|
||||
|
||||
# Specific operation schemas inherit from base
|
||||
class PostCreate(PostBase):
|
||||
pass # Only title and content needed for creation
|
||||
|
||||
class PostRead(PostBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: int
|
||||
created_at: datetime
|
||||
created_by_user_id: int
|
||||
is_deleted: bool = False # From model's soft delete fields
|
||||
```
|
||||
|
||||
**Purpose**: Reduces duplication and ensures consistency across related schemas.
|
||||
|
||||
### Optional Fields in Updates
|
||||
|
||||
```python
|
||||
class PostUpdate(BaseModel):
|
||||
title: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
min_length=1,
|
||||
max_length=100,
|
||||
default=None
|
||||
)
|
||||
]
|
||||
content: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
min_length=1,
|
||||
max_length=10000,
|
||||
default=None
|
||||
)
|
||||
]
|
||||
```
|
||||
|
||||
**Pattern**: All fields optional in update schemas. Only provided fields are updated in the database.
|
||||
|
||||
### Nested Schemas
|
||||
|
||||
```python
|
||||
# Post schema with user information
|
||||
class PostWithUser(PostRead):
|
||||
created_by_user: UserRead # Nested user data
|
||||
|
||||
# Alternative: Custom nested schema
|
||||
class PostAuthor(BaseModel):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: int
|
||||
username: str
|
||||
# Only include fields needed for this context
|
||||
|
||||
class PostRead(PostBase):
|
||||
created_by_user: PostAuthor
|
||||
```
|
||||
|
||||
**Usage**: Include related model data in responses without exposing all fields.
|
||||
|
||||
## Validation Patterns
|
||||
|
||||
### Custom Validators
|
||||
|
||||
```python
|
||||
from pydantic import field_validator, model_validator
|
||||
|
||||
class UserCreateWithConfirm(UserBase):
|
||||
password: str
|
||||
confirm_password: str
|
||||
|
||||
@field_validator('username')
|
||||
@classmethod
|
||||
def validate_username(cls, v):
|
||||
if v.lower() in ['admin', 'root', 'system']:
|
||||
raise ValueError('Username not allowed')
|
||||
return v.lower() # Normalize to lowercase
|
||||
|
||||
@model_validator(mode='after')
|
||||
def validate_passwords_match(self):
|
||||
if self.password != self.confirm_password:
|
||||
raise ValueError('Passwords do not match')
|
||||
return self
|
||||
```
|
||||
|
||||
**field_validator**: Validates individual fields. Can transform values.
|
||||
|
||||
**model_validator**: Validates across multiple fields. Access to full model data.
|
||||
|
||||
### Computed Fields
|
||||
|
||||
```python
|
||||
from pydantic import computed_field
|
||||
|
||||
class UserReadWithComputed(UserRead):
|
||||
created_at: datetime # Would need to be added to actual UserRead
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
def age_days(self) -> int:
|
||||
return (datetime.utcnow() - self.created_at).days
|
||||
|
||||
@computed_field
|
||||
@property
|
||||
def display_name(self) -> str:
|
||||
return f"@{self.username}"
|
||||
```
|
||||
|
||||
**Purpose**: Add computed values to API responses without storing them in the database.
|
||||
|
||||
### Conditional Validation
|
||||
|
||||
```python
|
||||
class PostCreate(BaseModel):
|
||||
title: str
|
||||
content: str
|
||||
category: Optional[str] = None
|
||||
is_premium: bool = False
|
||||
|
||||
@model_validator(mode='after')
|
||||
def validate_premium_content(self):
|
||||
if self.is_premium and not self.category:
|
||||
raise ValueError('Premium posts must have a category')
|
||||
return self
|
||||
```
|
||||
|
||||
## Schema Configuration
|
||||
|
||||
### Model Config Options
|
||||
|
||||
```python
|
||||
class UserRead(BaseModel):
|
||||
model_config = ConfigDict(
|
||||
from_attributes=True, # Allow creation from SQLAlchemy models
|
||||
extra="forbid", # Reject extra fields
|
||||
str_strip_whitespace=True, # Strip whitespace from strings
|
||||
validate_assignment=True, # Validate on field assignment
|
||||
populate_by_name=True, # Allow field names and aliases
|
||||
)
|
||||
```
|
||||
|
||||
### Field Aliases
|
||||
|
||||
```python
|
||||
class UserResponse(BaseModel):
|
||||
user_id: Annotated[
|
||||
int,
|
||||
Field(alias="id")
|
||||
]
|
||||
username: str
|
||||
email_address: Annotated[
|
||||
str,
|
||||
Field(alias="email")
|
||||
]
|
||||
|
||||
model_config = ConfigDict(populate_by_name=True)
|
||||
```
|
||||
|
||||
**Usage**: API can accept both `id` and `user_id`, `email` and `email_address`.
|
||||
|
||||
## Response Schema Patterns
|
||||
|
||||
### Multi-Record Responses
|
||||
|
||||
[FastCRUD's](https://benavlabs.github.io/fastcrud/) `get_multi` method returns a `GetMultiResponse`:
|
||||
|
||||
```python
|
||||
# Using get_multi directly
|
||||
users = await crud_users.get_multi(
|
||||
db=db,
|
||||
offset=0,
|
||||
limit=10,
|
||||
schema_to_select=UserRead,
|
||||
return_as_model=True,
|
||||
return_total_count=True
|
||||
)
|
||||
# Returns GetMultiResponse structure:
|
||||
# {
|
||||
# "data": [UserRead, ...],
|
||||
# "total_count": 150
|
||||
# }
|
||||
```
|
||||
|
||||
### Paginated Responses
|
||||
|
||||
For pagination with page numbers, use `PaginatedListResponse`:
|
||||
|
||||
```python
|
||||
from fastcrud.paginated import PaginatedListResponse
|
||||
|
||||
# In API endpoint - ONLY for paginated list responses
|
||||
@router.get("/users/", response_model=PaginatedListResponse[UserRead])
|
||||
async def get_users(page: int = 1, items_per_page: int = 10):
|
||||
# Returns paginated structure with additional pagination fields:
|
||||
# {
|
||||
# "data": [UserRead, ...],
|
||||
# "total_count": 150,
|
||||
# "has_more": true,
|
||||
# "page": 1,
|
||||
# "items_per_page": 10
|
||||
# }
|
||||
|
||||
# Single user endpoints return UserRead directly
|
||||
@router.get("/users/{user_id}", response_model=UserRead)
|
||||
async def get_user(user_id: int):
|
||||
# Returns single UserRead object:
|
||||
# {
|
||||
# "id": 1,
|
||||
# "name": "User Userson",
|
||||
# "username": "userson",
|
||||
# "email": "user.userson@example.com",
|
||||
# "profile_image_url": "https://...",
|
||||
# "tier_id": null
|
||||
# }
|
||||
```
|
||||
|
||||
### Error Response Schemas
|
||||
|
||||
```python
|
||||
class ErrorResponse(BaseModel):
|
||||
detail: str
|
||||
error_code: Optional[str] = None
|
||||
|
||||
class ValidationErrorResponse(BaseModel):
|
||||
detail: str
|
||||
errors: list[dict] # Pydantic validation errors
|
||||
```
|
||||
|
||||
### Success Response Wrapper
|
||||
|
||||
```python
|
||||
from typing import Generic, TypeVar
|
||||
|
||||
T = TypeVar('T')
|
||||
|
||||
class SuccessResponse(BaseModel, Generic[T]):
|
||||
success: bool = True
|
||||
data: T
|
||||
message: Optional[str] = None
|
||||
|
||||
# Usage in endpoint
|
||||
@router.post("/users/", response_model=SuccessResponse[UserRead])
|
||||
async def create_user(user_data: UserCreate):
|
||||
user = await crud_users.create(db=db, object=user_data)
|
||||
return SuccessResponse(data=user, message="User created successfully")
|
||||
```
|
||||
|
||||
## Creating New Schemas
|
||||
|
||||
### Step-by-Step Process
|
||||
|
||||
1. **Create schema file** in `src/app/schemas/your_model.py`
|
||||
2. **Define base schema** with common fields
|
||||
3. **Create operation-specific schemas** (Create, Read, Update, Delete)
|
||||
4. **Add validation rules** as needed
|
||||
5. **Import in __init__.py** for easy access
|
||||
|
||||
### Example: Category Schemas
|
||||
|
||||
```python
|
||||
# src/app/schemas/category.py
|
||||
from datetime import datetime
|
||||
from typing import Annotated
|
||||
from pydantic import BaseModel, Field, ConfigDict
|
||||
|
||||
class CategoryBase(BaseModel):
|
||||
name: Annotated[
|
||||
str,
|
||||
Field(
|
||||
min_length=1,
|
||||
max_length=50
|
||||
)
|
||||
]
|
||||
description: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
max_length=255,
|
||||
default=None
|
||||
)
|
||||
]
|
||||
|
||||
class CategoryCreate(CategoryBase):
|
||||
pass
|
||||
|
||||
class CategoryRead(CategoryBase):
|
||||
model_config = ConfigDict(from_attributes=True)
|
||||
|
||||
id: int
|
||||
created_at: datetime
|
||||
|
||||
class CategoryUpdate(BaseModel):
|
||||
name: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
min_length=1,
|
||||
max_length=50,
|
||||
default=None
|
||||
)
|
||||
]
|
||||
description: Annotated[
|
||||
str | None,
|
||||
Field(
|
||||
max_length=255,
|
||||
default=None
|
||||
)
|
||||
]
|
||||
|
||||
class CategoryWithPosts(CategoryRead):
|
||||
posts: list[PostRead] = [] # Include related posts
|
||||
```
|
||||
|
||||
### Import in __init__.py
|
||||
|
||||
```python
|
||||
# src/app/schemas/__init__.py
|
||||
from .user import UserCreate, UserRead, UserUpdate
|
||||
from .post import PostCreate, PostRead, PostUpdate
|
||||
from .category import CategoryCreate, CategoryRead, CategoryUpdate
|
||||
```
|
||||
|
||||
## Schema Testing
|
||||
|
||||
### Validation Testing
|
||||
|
||||
```python
|
||||
# tests/test_schemas.py
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
from app.schemas.user import UserCreate
|
||||
|
||||
def test_user_create_valid():
|
||||
user_data = {
|
||||
"name": "Test User",
|
||||
"username": "testuser",
|
||||
"email": "test@example.com",
|
||||
"password": "Str1ngst!"
|
||||
}
|
||||
user = UserCreate(**user_data)
|
||||
assert user.username == "testuser"
|
||||
assert user.name == "Test User"
|
||||
|
||||
def test_user_create_invalid_email():
|
||||
with pytest.raises(ValidationError) as exc_info:
|
||||
UserCreate(
|
||||
name="Test User",
|
||||
username="test",
|
||||
email="invalid-email",
|
||||
password="Str1ngst!"
|
||||
)
|
||||
|
||||
errors = exc_info.value.errors()
|
||||
assert any(error['type'] == 'value_error' for error in errors)
|
||||
|
||||
def test_password_validation():
|
||||
with pytest.raises(ValidationError) as exc_info:
|
||||
UserCreate(
|
||||
name="Test User",
|
||||
username="test",
|
||||
email="test@example.com",
|
||||
password="123" # Doesn't match pattern
|
||||
)
|
||||
```
|
||||
|
||||
### Serialization Testing
|
||||
|
||||
```python
|
||||
from app.models.user import User
|
||||
from app.schemas.user import UserRead
|
||||
|
||||
def test_user_read_from_model():
|
||||
# Create model instance
|
||||
user_model = User(
|
||||
id=1,
|
||||
name="Test User",
|
||||
username="testuser",
|
||||
email="test@example.com",
|
||||
profile_image_url="https://example.com/image.jpg",
|
||||
hashed_password="hashed123",
|
||||
is_superuser=False,
|
||||
tier_id=None,
|
||||
created_at=datetime.utcnow()
|
||||
)
|
||||
|
||||
# Convert to schema
|
||||
user_schema = UserRead.model_validate(user_model)
|
||||
assert user_schema.username == "testuser"
|
||||
assert user_schema.id == 1
|
||||
assert user_schema.name == "Test User"
|
||||
# hashed_password not included in UserRead
|
||||
```
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
### Model vs Schema Field Names
|
||||
|
||||
```python
|
||||
# DON'T - Exposing sensitive fields
|
||||
class UserRead(BaseModel):
|
||||
hashed_password: str # Never expose password hashes
|
||||
|
||||
# DO - Only expose safe fields
|
||||
class UserRead(BaseModel):
|
||||
id: int
|
||||
name: str
|
||||
username: str
|
||||
email: str
|
||||
profile_image_url: str
|
||||
tier_id: int | None
|
||||
```
|
||||
|
||||
### Validation Performance
|
||||
|
||||
```python
|
||||
# DON'T - Complex validation in every request
|
||||
@field_validator('email')
|
||||
@classmethod
|
||||
def validate_email_unique(cls, v):
|
||||
# Database query in validator - slow!
|
||||
if crud_users.exists(email=v):
|
||||
raise ValueError('Email already exists')
|
||||
|
||||
# DO - Handle uniqueness in business logic
|
||||
# Let database unique constraints handle this
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
Now that you understand schema implementation:
|
||||
|
||||
1. **[CRUD Operations](crud.md)** - Learn how schemas integrate with database operations
|
||||
2. **[Migrations](migrations.md)** - Manage database schema changes
|
||||
3. **[API Endpoints](../api/endpoints.md)** - Use schemas in FastAPI endpoints
|
||||
|
||||
The next section covers CRUD operations and how they use these schemas for data validation and transformation.
|
||||
Reference in New Issue
Block a user