# Phase 12: Attachment Research & Backend Validation - Research

**Researched:** 2026-02-02
**Domain:** File Upload & Chat Integration with Existing Rails Backend
**Confidence:** HIGH

## Summary

This phase validates that the backend infrastructure for file attachments already exists and is fully functional. The `MusicNotation` model in Rails handles file storage to S3 via CarrierWave, the `ApiMusicNotationsController` provides REST endpoints for upload/download/delete, and the `ChatMessage` model has built-in support for attachment associations. The legacy CoffeeScript `AttachmentStore` demonstrates the complete upload flow including client-side validation, FormData construction, and integration with chat message creation.

The existing infrastructure requires NO backend changes. The task is to port the upload/validation logic to React and integrate with the existing jam-ui chat window. The backend automatically creates chat messages with attachment references when files are uploaded, and broadcasts them via WebSocket with attachment metadata (id, type, name).

**Primary recommendation:** Reuse the existing `/api/music_notations` endpoint exactly as the legacy client does - multipart FormData upload with session_id and attachment_type. Focus implementation on React file input handling, client-side validation, and rendering attachment messages in JKChatMessage.

## Standard Stack

The established libraries/tools for this domain:

### Core (Already Exists - Backend)
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| CarrierWave | (Rails gem) | File uploads to S3 | Already configured in MusicNotationUploader |
| AWS SDK | (Rails gem) | S3 storage | Already configured with signed URLs |
| Rails 4.2.8 | 4.2.8 | Backend API | Existing infrastructure |

### Core (jam-ui Frontend)
| Library | Version | Purpose | Why Standard |
|---------|---------|---------|--------------|
| React | 16.13.1 | UI components | Already in use |
| Redux Toolkit | 1.6.1 | State management | Already in use for sessionChatSlice |
| Native File API | Browser | File selection | No external library needed |
| FormData | Browser | Multipart uploads | Standard web API |

### Supporting
| Library | Version | Purpose | When to Use |
|---------|---------|---------|-------------|
| uuid | v1 (already installed) | Unique IDs | For optimistic updates |
| PropTypes | (already installed) | Type checking | Component props validation |

### Alternatives Considered
| Instead of | Could Use | Tradeoff |
|------------|-----------|----------|
| Native File API | react-dropzone | Adds complexity, drag-drop not in requirements |
| Direct S3 upload | Backend proxy | Backend already handles S3, signed URLs on download |
| Custom progress | XMLHttpRequest | Fetch doesn't support upload progress, but not required per reqs |

**Installation:**
```bash
# No new dependencies required
# All infrastructure already exists
```

## Architecture Patterns

### Existing Backend API Surface

**POST /api/music_notations** - Upload files
```
Request: multipart/form-data
- files[]: File (multiple allowed)
- session_id: string (music session ID)
- attachment_type: 'notation' | 'audio'

Response: 201 Created
[
  {
    "id": "uuid",
    "file_name": "document.pdf",
    "file_url": "/api/music_notations/{id}"
  }
]

Errors:
- 413: File too large (server-side check)
- 422: Validation errors (type, other)
```

**GET /api/music_notations/:id** - Get signed download URL
```
Response: 200 OK
{
  "url": "https://s3.amazonaws.com/...?signature=..."
}
```

**DELETE /api/music_notations/:id** - Delete attachment
```
Response: 204 No Content
Errors: 403 if not authorized
```

### Existing File Validation (from MusicNotationUploader)

**Allowed Extensions:**
```ruby
def extension_white_list
  %w(pdf png jpg jpeg gif xml mxl txt wav flac ogg aiff aifc au)
```

Note: Requirements specify `.mp3, .wav` but backend has `.wav, .flac, .ogg, .aiff, .aifc, .au`. May need backend update OR client-side restriction to match requirements exactly.

**Client-Side Size Limit (from AttachmentStore):**
```javascript
max = 10 * 1024 * 1024; // 10 MB
if (file.size > max) {
  // Show error, reject file
}
```

### WebSocket Chat Message with Attachment

The `ChatMessage.send_chat_msg` method broadcasts attachment info via WebSocket:
```ruby
msg = @@message_factory.chat_message(
  music_session_id,
  user.name,
  user.id,
  chat_msg.message,
  chat_msg.id,
  chat_msg.created_at.utc.iso8601,
  channel,
  lesson_session_id,
  purpose,              # 'Notation File' or 'Audio File'
  attachment_id,        # music_notation.id
  attachment_type,      # 'notation' or 'audio'
  attachment_name       # music_notation.file_name
)
```

### Recommended Project Structure

```
jam-ui/src/
├── components/
│   └── client/
│       └── chat/
│           ├── JKChatComposer.js       # Modify: Add attach button
│           ├── JKChatMessage.js        # Modify: Handle attachment display
│           └── JKChatAttachButton.js   # NEW: Hidden file input + trigger button
├── services/
│   └── attachmentService.js            # NEW: Upload logic, validation
└── store/features/
    └── sessionChatSlice.js             # Modify: Upload state management
```

### Pattern 1: File Upload Flow
**What:** Client-side validation followed by multipart FormData upload
**When to use:** When user selects file via attach button
**Example:**
```javascript
// Source: Legacy AttachmentStore.js.coffee translated
export const uploadAttachment = async (file, sessionId) => {
  // 1. Client-side validation
  const MAX_SIZE = 10 * 1024 * 1024;
  if (file.size > MAX_SIZE) {
    throw new Error('File exceeds 10 MB limit');
  }

  const ALLOWED_TYPES = ['.pdf', '.xml', '.mxl', '.txt', '.png', '.jpg', '.jpeg', '.gif', '.mp3', '.wav'];
  const ext = '.' + file.name.split('.').pop().toLowerCase();
  if (!ALLOWED_TYPES.includes(ext)) {
    throw new Error(`File type ${ext} not allowed`);
  }

  // 2. Build FormData (exactly like legacy)
  const formData = new FormData();
  formData.append('files[]', file);
  formData.append('session_id', sessionId);
  formData.append('attachment_type', isAudioType(ext) ? 'audio' : 'notation');

  // 3. Upload via fetch (NOT apiFetch - needs different headers)
  const response = await fetch(`${API_BASE}/music_notations`, {
    method: 'POST',
    credentials: 'include',
    // Do NOT set Content-Type - browser sets it with boundary for FormData
    body: formData
  });

  if (!response.ok) {
    if (response.status === 413) {
      throw new Error('File too large - maximum 10 MB');
    }
    throw new Error('Upload failed');
  }

  return response.json();
};
```

### Pattern 2: Attachment Message Display
**What:** Render attachment link in chat message
**When to use:** When message has attachment metadata
**Example:**
```javascript
// In JKChatMessage.js
const JKChatMessage = ({ message }) => {
  const handleAttachmentClick = async (attachmentId) => {
    // Fetch signed URL then open
    const response = await fetch(`${API_BASE}/music_notations/${attachmentId}`, {
      credentials: 'include'
    });
    const { url } = await response.json();
    window.open(url, '_blank');
  };

  // Check for attachment (from WebSocket message or history)
  if (message.attachmentId) {
    return (
      <div className="chat-message attachment">
        <span>{message.senderName} attached a file:</span>
        <a onClick={() => handleAttachmentClick(message.attachmentId)}>
          {message.attachmentName}
        </a>
      </div>
    );
  }

  // Regular text message...
};
```

### Anti-Patterns to Avoid
- **Setting Content-Type for FormData:** Browser must set it with multipart boundary
- **Using apiFetch for uploads:** It forces JSON Content-Type, breaks FormData
- **Storing File objects in Redux:** Not serializable, causes issues
- **Direct S3 uploads from client:** Backend already handles this, no need to change

## Don't Hand-Roll

Problems that look simple but have existing solutions:

| Problem | Don't Build | Use Instead | Why |
|---------|-------------|-------------|-----|
| S3 signed URLs | Custom S3 signing | Backend `/api/music_notations/:id` | Backend already has AWS SDK configured |
| File type validation | Custom regex | Whitelist from requirements | Exact types specified, match them |
| Chat message creation | Custom chat POST with attachment | Let upload endpoint do it | Backend creates ChatMessage automatically |
| WebSocket attachment broadcast | Custom message sending | Existing ws infrastructure | Backend sends CHAT_MESSAGE with attachment fields |

**Key insight:** The backend is fully functional. Zero backend changes needed. Focus entirely on React UI/UX.

## Common Pitfalls

### Pitfall 1: FormData Content-Type Header
**What goes wrong:** Setting `Content-Type: 'application/json'` or `Content-Type: 'multipart/form-data'` manually
**Why it happens:** Habit from other API calls, or thinking you need to specify multipart
**How to avoid:** Do NOT set Content-Type header when using FormData - browser adds it with boundary
**Warning signs:** "Unexpected end of input" errors, "boundary not found" errors

### Pitfall 2: File Type Mismatch Between Requirements and Backend
**What goes wrong:** Requirements say `.mp3` but backend whitelist doesn't include it
**Why it happens:** Backend whitelist was written before requirements finalized
**How to avoid:** Validate client-side against REQUIREMENTS list, update backend whitelist if needed
**Warning signs:** User uploads .mp3, backend rejects it

### Pitfall 3: Not Waiting for Chat Message via WebSocket
**What goes wrong:** Trying to show attachment in chat immediately after upload response
**Why it happens:** Assuming upload response contains full chat message
**How to avoid:** Let WebSocket deliver the chat message with attachment; backend creates it
**Warning signs:** Duplicate messages, missing attachment metadata in UI

### Pitfall 4: Blocking UI During Upload
**What goes wrong:** User can't interact with chat during file upload
**Why it happens:** Not handling async state properly
**How to avoid:** Use Redux upload status, show progress indicator, keep UI responsive
**Warning signs:** Frozen UI, users clicking multiple times

### Pitfall 5: Missing Error Handling for 413
**What goes wrong:** Generic error shown when file too large
**Why it happens:** Not checking response.status before generic error
**How to avoid:** Check for 413 specifically, show user-friendly "file too large" message
**Warning signs:** Users don't understand why upload failed

## Code Examples

Verified patterns from official sources:

### FormData File Upload (from legacy AttachmentStore)
```javascript
// Source: web/app/assets/javascripts/react-components/stores/AttachmentStore.js.coffee
// Translated to modern JavaScript

const uploadMusicNotations = (formData) => {
  return fetch('/api/music_notations', {
    method: 'POST',
    processData: false,    // jQuery equivalent: don't process
    // contentType: false,  // Don't set - let browser handle
    credentials: 'include',
    body: formData
  });
};

// Usage:
const formData = new FormData();
formData.append('files[]', file);
formData.append('session_id', sessionId);
formData.append('attachment_type', 'notation'); // or 'audio'
```

### Attachment Download (from legacy ChatWindow)
```javascript
// Source: web/app/assets/javascripts/react-components/ChatWindow.js.jsx.coffee
// Translated to modern JavaScript

const handleAttachmentClick = async (attachmentId) => {
  const response = await fetch(`/api/music_notations/${attachmentId}`, {
    credentials: 'include'
  });
  const { url } = await response.json();
  window.open(url, '_blank');
};
```

### File Input Trigger Pattern
```javascript
// Hidden input with button trigger (common React pattern)
const AttachButton = ({ onFileSelect }) => {
  const inputRef = useRef(null);

  return (
    <>
      <button onClick={() => inputRef.current?.click()}>
        Attach File
      </button>
      <input
        ref={inputRef}
        type="file"
        style={{ display: 'none' }}
        accept=".pdf,.xml,.mxl,.txt,.png,.jpg,.jpeg,.gif,.mp3,.wav"
        onChange={(e) => {
          if (e.target.files?.[0]) {
            onFileSelect(e.target.files[0]);
            e.target.value = ''; // Reset for same file re-selection
          }
        }}
      />
    </>
  );
};
```

## State of the Art

| Old Approach | Current Approach | When Changed | Impact |
|--------------|------------------|--------------|--------|
| jQuery $.ajax | fetch API | jam-ui migration | All new code uses fetch |
| CoffeeScript | JavaScript/ES6 | jam-ui creation | New components in JS |
| Reflux stores | Redux Toolkit | jam-ui migration | State in sessionChatSlice |

**Deprecated/outdated:**
- AttachmentStore.js.coffee: Reference only, don't port directly
- jQuery file handling: Use native File API

## Integration Points Summary

### 1. Chat Composer (JKChatComposer.js)
**Current:** Text input + Send button
**Add:** Attach button next to Send button
**State:** uploading flag in sessionChatSlice

### 2. Chat Message (JKChatMessage.js)
**Current:** Shows sender + text + timestamp
**Add:** Handle attachment type messages (purpose = 'Notation File' or 'Audio File')
**Data:** message.attachmentId, message.attachmentType, message.attachmentName

### 3. WebSocket Handler (JKSessionScreen.js)
**Current:** handleChatMessage transforms payload to Redux format
**Add:** Include attachment fields in transformation
```javascript
const message = {
  id: payload.msg_id,
  // ... existing fields
  purpose: payload.purpose,           // 'Notation File', 'Audio File', or undefined
  attachmentId: payload.attachment_id,
  attachmentType: payload.attachment_type,
  attachmentName: payload.attachment_name
};
```

### 4. REST Helper (rest.js)
**Add:** New function for multipart upload
```javascript
export const uploadMusicNotation = (formData) => {
  const baseUrl = process.env.REACT_APP_API_BASE_URL;
  return fetch(`${baseUrl}/music_notations`, {
    method: 'POST',
    credentials: 'include',
    body: formData
  });
};

export const getMusicNotationUrl = (id) => {
  return apiFetch(`/music_notations/${id}`);
};
```

## Open Questions

Things that couldn't be fully resolved:

1. **File Type Whitelist Sync**
   - What we know: Backend allows `pdf png jpg jpeg gif xml mxl txt wav flac ogg aiff aifc au`
   - What's unclear: Requirements specify `mp3` but backend doesn't allow it; requirements don't mention `flac ogg aiff aifc au`
   - Recommendation: Use requirements list client-side, verify backend supports `.mp3` (may need backend update)

2. **Progress Indicator Implementation**
   - What we know: Requirements want progress indicator (REQ-1.4)
   - What's unclear: fetch() doesn't support upload progress; XMLHttpRequest does
   - Recommendation: Use simple "Uploading..." spinner; if real progress needed, use XMLHttpRequest

3. **Multiple File Upload**
   - What we know: Backend supports `files[]` array, legacy allowed multiple
   - What's unclear: Requirements don't specify single vs multiple file selection
   - Recommendation: Implement single file for v1, array support in code for easy extension

## Sources

### Primary (HIGH confidence)
- `/Users/nuwan/Code/jam-cloud/web/app/controllers/api_music_notations_controller.rb` - Backend endpoint implementation
- `/Users/nuwan/Code/jam-cloud/ruby/lib/jam_ruby/models/music_notation.rb` - Model with S3 integration
- `/Users/nuwan/Code/jam-cloud/ruby/lib/jam_ruby/models/chat_message.rb` - Chat-attachment association
- `/Users/nuwan/Code/jam-cloud/web/app/assets/javascripts/react-components/stores/AttachmentStore.js.coffee` - Legacy upload flow
- `/Users/nuwan/Code/jam-cloud/jam-ui/src/store/features/sessionChatSlice.js` - Existing chat Redux slice
- `/Users/nuwan/Code/jam-cloud/jam-ui/src/components/client/chat/` - Existing chat components

### Secondary (MEDIUM confidence)
- `/Users/nuwan/Code/jam-cloud/ruby/lib/jam_ruby/message_factory.rb` - WebSocket message structure with attachment fields
- `/Users/nuwan/Code/jam-cloud/pb/src/client_container.proto` - Protocol buffer definition for ChatMessage

### Tertiary (LOW confidence)
- None - all sources are primary codebase files

## Metadata

**Confidence breakdown:**
- Standard stack: HIGH - Existing infrastructure fully documented in codebase
- Architecture: HIGH - Clear patterns from legacy implementation to port
- Pitfalls: HIGH - Based on actual legacy code issues and common patterns

**Research date:** 2026-02-02
**Valid until:** 2026-03-02 (stable - backend unlikely to change)
