Файловый обмен - Upload File

Возможные требования по передаче файлов:

• Требования по передаче файлов
• Multiple file upload
• Validation and security
• Upload from remote server (FTP, HTTP…)
• Upload from private storage bucket (S3 or Google Storage)
• Individual progress bar for every file
• Upload notifications
• Разделение на части файла - Chunking files over 20 MB
• Parallel upload of files under 20 MB

Patterns

• Image API service dedicated
• Signed Upload URL dedicated storage (On premise, Cloud)
  • It can require a little more work to set up than a dedicated image service but provides for finer control.
• Google Resumable upload large files > 5Mb
• Legacy: Несколько файлов multipart/form-data, and return an ID to the client.
  1. The client then sends (PUT idempotent) the metadata with the ID, and the server re-associates the file and the metadata.
  2. Send the metadata first, and return an ID to the client. The client then sends the file with the ID, and the server re-associates the file and the metadata.
  3. CSRF attacks risk
     1. You should not enable multipart requests (ie, graphql-upload) in your GraphQL server unless you understand how CSRF attacks work and are confident that you have prevented them in your app
     2. exposes your server to CSRF mutation attacks unless you’ve specifically prevented them.
• Legacy: include the Base64 content inside the JSON string
  • Не использовать base64 кодирования: increasing the data size by around 33%
  • add processing overhead in both the server and the client for encoding/decoding.
• Несколько файлов разных типов multipart/related request.
  • The Multipart/Related media type is intended for compound objects consisting of several inter-related body parts. RFC 2387 specification for more in-depth details.
• Option
  • Direct file upload
    • Create a new file, POST /downloads (endpoint name an example), response 202 Accepted, Location: /downloads/123 (unique ID of the download). If we’re sharing the files among users, the same POST might return the same ID (all the users are waiting for the same file to generate)
  • Multipart HTTP request
  • Two-step metadata + upload
• HTTP Status
  • 413 Payload Too Large (if exceeding max)
  • 507 Insufficient Storage (server full).
• API Uploading example Cloudinary

Security

• OWASP
  • OWASP Unrestricted File Upload
• OWASP TOP 10
• OWASP Cheats Sheets

Технологии

• Файловые системы
  • Распределенные файловые системы
  • Распределенные параллельные файловые системы
• FTP SFTP - передача файлов
• Object Storage