Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MaxBytes Setting Issue in go-zero Framework's httpx Package #4679

Open
zzf-zhu-Miracle opened this issue Mar 3, 2025 · 1 comment
Open
Labels
area/api Categorizes issue or PR as related to API syntax or usages.

Comments

@zzf-zhu-Miracle
Copy link

MaxBytes Setting Issue in go-zero Framework's httpx Package

Is your feature request related to a problem? Please describe.
When using the go-zero framework to handle large file uploads, I encountered request body size limitations. Currently, the Parse function in the httpx package returns an "unexpected EOF" error when processing large request bodies (such as base64-encoded PDF files) because the request body exceeds the default size limit.

Describe the solution you'd like
I would like the httpx package to provide a public API for setting the maximum request body size (MaxBytes). While it's possible to manually set this using http.MaxBytesReader, having a unified configuration method at the framework level would be more convenient.

I suggest adding a function to the httpx package, such as SetMaxBytes or ParseWithMaxBytes, allowing developers to specify the maximum request body size when calling the Parse function.

// Suggested API
func ParseWithMaxBytes(r *http.Request, v interface{}, maxBytes int64) error {
    // Implementation code
}

// Or provide a configuration option
func Parse(r *http.Request, v interface{}, opts ...ParseOption) error {
    // Implementation code
}

type ParseOption func(*parseOptions)

func WithMaxBytes(maxBytes int64) ParseOption {
    return func(o *parseOptions) {
        o.maxBytes = maxBytes
    }
}

Describe alternatives you've considered

  1. Using http.MaxBytesReader to manually set the request body size limit
  2. Setting MaxBytes globally in the server configuration
  3. Using streaming parsing or chunked uploads

While these methods can solve the problem, providing a unified API at the framework level would be more elegant and consistent.

Additional context
Currently, we must manually add the following code in each handler that processes large requests:

r.Body = http.MaxBytesReader(w, r.Body, maxRequestSize)

Then use json.Unmarshal instead of httpx.Parse to parse the request body, which breaks the framework's consistency.

If the httpx package could provide this functionality, it would greatly simplify the code for handling large requests while maintaining framework consistency.

@kevwan
Copy link
Contributor

kevwan commented Mar 6, 2025

Typically, httpx is not used for file uploading. I recommend you to use native http package to handle this kind of requests.

@kevwan kevwan added the area/api Categorizes issue or PR as related to API syntax or usages. label Mar 6, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/api Categorizes issue or PR as related to API syntax or usages.
Projects
None yet
Development

No branches or pull requests

2 participants