乐闻世界logo
搜索文章和话题

服务端面试题手册

GraphQL 查询、变更和订阅有什么区别

GraphQL 查询(Query)、变更(Mutation)和订阅(Subscription)的区别GraphQL 提供了三种主要的操作类型,每种类型都有特定的用途和语义。1. 查询(Query)定义: 用于获取数据,类似于 REST 中的 GET 请求。特点:只读操作,不会修改服务器上的数据可以并行执行多个查询可以嵌套查询以获取相关数据可以使用参数来过滤或定制结果示例:query GetUser($userId: ID!) { user(id: $userId) { id name email posts { id title createdAt } }}使用场景:获取用户信息列表查询数据详情展示报表生成2. 变更(Mutation)定义: 用于修改服务器上的数据,类似于 REST 中的 POST、PUT、DELETE 请求。特点:会修改服务器上的数据按顺序执行(串行),确保数据一致性通常返回修改后的数据或操作结果可以包含输入对象和参数示例:mutation CreateUser($input: CreateUserInput!) { createUser(input: $input) { id name email createdAt }}mutation UpdateUser($userId: ID!, $input: UpdateUserInput!) { updateUser(id: $userId, input: $input) { id name email updatedAt }}mutation DeleteUser($userId: ID!) { deleteUser(id: $userId) { success message }}使用场景:创建新资源更新现有资源删除资源批量操作3. 订阅(Subscription)定义: 用于实时数据推送,当服务器端数据发生变化时,客户端会自动收到更新。特点:建立持久连接(通常使用 WebSocket)服务器主动推送数据到客户端适用于实时性要求高的场景可以包含过滤条件示例:subscription OnUserCreated { userCreated { id name email createdAt }}subscription OnPostUpdated($postId: ID!) { postUpdated(postId: $postId) { id title content updatedAt }}使用场景:聊天应用实时通知实时数据监控协作编辑核心区别对比| 特性 | Query | Mutation | Subscription ||------|-------|----------|--------------|| 数据修改 | 只读 | 修改数据 | 只读 || 执行方式 | 并行 | 串行 | 持久连接 || 实时性 | 按需请求 | 按需请求 | 实时推送 || 网络协议 | HTTP | HTTP | WebSocket || 缓存 | 可缓存 | 不可缓存 | 不可缓存 || 幂等性 | 幂等 | 非幂等 | 非幂等 |最佳实践Query 最佳实践避免过度获取,只请求需要的字段使用参数进行数据过滤合理使用分页利用 GraphQL 的类型系统进行数据验证Mutation 最佳实践使用输入对象(Input Types)封装参数返回修改后的完整数据提供清晰的成功/失败响应实现适当的错误处理和验证Subscription 最佳实践提供过滤条件减少不必要的数据推送实现连接管理和重连机制考虑使用负载均衡处理大量订阅设置合理的超时和心跳机制性能考虑Query: 使用 DataLoader 解决 N+1 查询问题Mutation: 使用事务确保数据一致性Subscription: 使用消息队列处理高并发订阅
阅读 0·2月21日 17:00

GraphQL 性能优化有哪些策略

GraphQL 性能优化策略GraphQL 的灵活性虽然强大,但也可能带来性能挑战。以下是优化 GraphQL API 性能的关键策略。1. 解决 N+1 查询问题问题描述当查询嵌套关系时,每个父对象都会触发一次子对象的查询,导致大量数据库查询。解决方案:DataLoaderconst DataLoader = require('dataloader');// 创建 User 的 DataLoaderconst userLoader = new DataLoader(async (userIds) => { const users = await User.findAll({ where: { id: userIds } }); // 按照请求的顺序返回结果 return userIds.map(id => users.find(user => user.id === id));});// 在 Resolver 中使用const resolvers = { Post: { author: (post) => userLoader.load(post.authorId) }};优势:批量查询,减少数据库往返自动去重和缓存保持查询顺序2. 查询复杂度分析限制查询深度const depthLimit = require('graphql-depth-limit');const server = new ApolloServer({ typeDefs, resolvers, validationRules: [ depthLimit(7) // 限制查询深度为 7 层 ]});限制查询复杂度const { createComplexityLimitRule } = require('graphql-validation-complexity');const complexityLimitRule = createComplexityLimitRule(1000, { onCost: (cost) => console.log(`Query cost: ${cost}`)});const server = new ApolloServer({ typeDefs, resolvers, validationRules: [complexityLimitRule]});3. 字段级缓存使用 Redis 缓存const Redis = require('ioredis');const redis = new Redis();async function cachedResolver(parent, args, context, info) { const cacheKey = `graphql:${info.fieldName}:${JSON.stringify(args)}`; // 尝试从缓存获取 const cached = await redis.get(cacheKey); if (cached) { return JSON.parse(cached); } // 执行实际查询 const result = await fetchData(args); // 缓存结果(5 分钟过期) await redis.setex(cacheKey, 300, JSON.stringify(result)); return result;}使用 Apollo Client 缓存const client = new ApolloClient({ cache: new InMemoryCache({ typePolicies: { Query: { fields: { posts: { keyArgs: ['filter'], merge(existing, incoming) { return incoming; } } } } } })});4. 查询持久化使用 persisted queriesconst { PersistedQueryLink } = require('@apollo/client/link/persisted-queries');const { createPersistedQueryLink } = require('@apollo/client/link/persisted-queries');const { sha256 } = require('crypto-hash');const link = createPersistedQueryLink({ sha256, useGETForHashedQueries: true });const client = new ApolloClient({ link: link.concat(httpLink), cache: new InMemoryCache()});优势:减少网络传输提高安全性降低服务器负载5. 数据库优化使用索引// 为常用查询字段添加索引User.addIndex('email');Post.addIndex(['authorId', 'createdAt']);优化关联查询// 使用 JOIN 而不是多次查询const postsWithAuthors = await Post.findAll({ include: [{ model: User, as: 'author', attributes: ['id', 'name', 'email'] }]});6. 分页优化使用游标分页type PageInfo { hasNextPage: Boolean! hasPreviousPage: Boolean! startCursor: String endCursor: String}type PostConnection { edges: [PostEdge!]! pageInfo: PageInfo! totalCount: Int!}type PostEdge { node: Post! cursor: String!}type Query { posts(after: String, first: Int, before: String, last: Int): PostConnection!}优势:性能稳定,不受数据量影响支持实时数据更新更好的用户体验7. 批量操作批量查询# 不好的做法 - 多次查询query { user1: user(id: "1") { name } user2: user(id: "2") { name } user3: user(id: "3") { name }}# 好的做法 - 批量查询query { users(ids: ["1", "2", "3"]) { id name }}批量变更mutation { createPosts(input: [ { title: "Post 1", content: "Content 1" }, { title: "Post 2", content: "Content 2" }, { title: "Post 3", content: "Content 3" } ]) { id title }}8. 懒加载使用 @defer 指令query GetUser($userId: ID!) { user(id: $userId) { id name email ... @defer { posts { id title } } }}优势:优先加载关键数据提高首屏渲染速度改善用户体验9. 订阅优化使用消息队列const { PubSub } = require('graphql-subscriptions');const RedisPubSub = require('graphql-redis-subscriptions').RedisPubSub;const pubsub = new RedisPubSub({ connection: { host: 'localhost', port: 6379 }});const POST_UPDATED = 'POST_UPDATED';const resolvers = { Mutation: { updatePost: (_, { id, input }) => { const updatedPost = updatePost(id, input); pubsub.publish(POST_UPDATED, { postUpdated: updatedPost }); return updatedPost; } }, Subscription: { postUpdated: { subscribe: () => pubsub.asyncIterator([POST_UPDATED]) } }};10. 监控和分析使用 Apollo Studioconst { ApolloServerPluginUsageReporting } = require('apollo-server-core');const server = new ApolloServer({ typeDefs, resolvers, plugins: [ ApolloServerPluginUsageReporting({ apiKey: process.env.APOLLO_KEY, graphRef: 'my-graph@current' }) ]});自定义监控const resolvers = { Query: { user: async (_, { id }, context) => { const startTime = Date.now(); try { const user = await User.findById(id); const duration = Date.now() - startTime; // 记录查询性能 context.metrics.recordQuery('user', duration); return user; } catch (error) { context.metrics.recordError('user', error); throw error; } } }};11. 性能优化检查清单[ ] 使用 DataLoader 解决 N+1 查询问题[ ] 实施查询深度和复杂度限制[ ] 配置适当的缓存策略[ ] 使用查询持久化[ ] 优化数据库查询和索引[ ] 实现高效的分页[ ] 支持批量操作[ ] 使用懒加载指令[ ] 优化订阅性能[ ] 设置监控和分析工具[ ] 定期进行性能测试[ ] 优化网络传输(压缩、HTTP/2)12. 常见性能问题及解决方案| 问题 | 原因 | 解决方案 ||------|------|----------|| 查询响应慢 | N+1 查询 | 使用 DataLoader || 数据库负载高 | 过度获取数据 | 限制查询字段,使用分页 || 内存占用高 | 缓存策略不当 | 设置合理的缓存过期时间 || 网络传输慢 | 查询过大 | 使用查询持久化,启用压缩 || 订阅延迟高 | 消息队列性能差 | 使用高性能消息队列(Redis) |
阅读 0·2月21日 17:00

GraphQL 有哪些重要的工具和生态系统

GraphQL 工具和生态系统详解GraphQL 拥有丰富的工具和生态系统,这些工具可以大大提高开发效率。以下是 GraphQL 生态系统中最重要的工具和库。1. 服务器框架Apollo Serverconst { ApolloServer } = require('apollo-server');const typeDefs = ` type Query { hello: String! }`;const resolvers = { Query: { hello: () => 'Hello World!' }};const server = new ApolloServer({ typeDefs, resolvers });server.listen().then(({ url }) => { console.log(`Server ready at ${url}`);});特性:开箱即用的 GraphQL 服务器内置 GraphQL Playground支持订阅、文件上传等丰富的插件系统集成 Apollo StudioGraphQL Yogaimport { createServer } from 'graphql-yoga';const server = createServer({ schema: { typeDefs: ` type Query { hello: String! } `, resolvers: { Query: { hello: () => 'Hello World!' } } }});server.start().then(() => { console.log('Server is running on http://localhost:4000');});特性:轻量级、高性能支持 Express、Fastify 等内置 WebSocket 支持支持 GraphQL 文件上传兼容 Apollo Server2. 客户端库Apollo Clientimport { ApolloClient, InMemoryCache, HttpLink } from '@apollo/client';const client = new ApolloClient({ link: new HttpLink({ uri: 'https://api.example.com/graphql' }), cache: new InMemoryCache()});特性:功能强大的缓存系统支持查询、变更、订阅乐观更新分页支持开发工具集成Relayimport { RelayEnvironment, RecordSource, Store, Network } from 'relay-runtime';const network = Network.create((operation, variables) => { return fetch('https://api.example.com/graphql', { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ query: operation.text, variables }) }).then(response => response.json());});const environment = new RelayEnvironment({ network, store: new Store(new RecordSource())});特性:Facebook 开发高度优化的数据获取自动数据规范化强类型查询适合大型应用URQLimport { createClient, Provider } from 'urql';import { cacheExchange, fetchExchange } from '@urql/core';const client = createClient({ url: 'https://api.example.com/graphql', exchanges: [cacheExchange, fetchExchange]});特性:轻量级(7KB)简单的 API可扩展的交换系统良好的 TypeScript 支持活跃的社区3. 开发工具GraphQL Code Generator# 安装npm install @graphql-codegen/cli# 配置文件 codegen.ymlschema: https://api.example.com/graphqldocuments: ./src/**/*.graphqlgenerates: ./src/generated/graphql.ts: plugins: - typescript - typescript-operations - typescript-react-apollo config: withHooks: true功能:从 Schema 生成 TypeScript 类型生成 React Hooks生成 Resolvers 类型支持多种框架自动保持类型同步GraphQL Playgroundconst { ApolloServer } = require('apollo-server');const server = new ApolloServer({ typeDefs, resolvers, playground: true, // 启用 Playground playgroundOptions: { endpoint: '/graphql', settings: { 'editor.theme': 'dark' } }});功能:交互式 GraphQL IDE实时查询执行自动完成和文档查询历史支持多个端点Apollo Studioconst { ApolloServerPluginUsageReporting } = require('apollo-server-core');const server = new ApolloServer({ typeDefs, resolvers, plugins: [ ApolloServerPluginUsageReporting({ apiKey: process.env.APOLLO_KEY, graphRef: 'my-graph@current' }) ]});功能:追踪查询性能分析 Schema 使用情况监控错误率实时告警Schema 变更管理4. 测试工具GraphQL Testing Libraryimport { render, screen } from '@testing-library/react';import { MockedProvider } from '@apollo/client/testing';import { GET_USERS } from './queries';import UserList from './UserList';const mocks = [ { request: { query: GET_USERS }, result: { data: { users: [ { id: '1', name: 'John' }, { id: '2', name: 'Jane' } ] } } }];test('renders user list', () => { render( <MockedProvider mocks={mocks}> <UserList /> </MockedProvider> ); expect(screen.getByText('John')).toBeInTheDocument(); expect(screen.getByText('Jane')).toBeInTheDocument();});Apollo Server Testingimport { ApolloServer } from 'apollo-server';import { createTestClient } from 'apollo-server-testing';const server = new ApolloServer({ typeDefs, resolvers });const { query, mutate } = createTestClient(server);test('should return users', async () => { const { data, errors } = await query({ query: 'query { users { id name } }' }); expect(errors).toBeUndefined(); expect(data.users).toBeDefined();});5. 性能优化工具DataLoaderconst DataLoader = require('dataloader');const userLoader = new DataLoader(async (userIds) => { const users = await User.findAll({ where: { id: userIds } }); return userIds.map(id => users.find(user => user.id === id));});const resolvers = { Post: { author: (post) => userLoader.load(post.authorId) }};功能:批量查询自动去重缓存结果解决 N+1 查询问题GraphQL Cache Controltype Query { user(id: ID!): User @cacheControl(maxAge: 300) posts: [Post!]! @cacheControl(maxAge: 60)}功能:声明式缓存控制CDN 友好减少服务器负载提高响应速度6. 安全工具GraphQL Shieldconst { shield, rule } = require('graphql-shield');const isAuthenticated = rule()((parent, args, context) => { return !!context.user;});const isAdmin = rule()((parent, args, context) => { return context.user?.role === 'ADMIN';});const permissions = shield({ Query: { me: isAuthenticated, users: isAdmin }, Mutation: { createUser: isAuthenticated, deleteUser: isAdmin }});const server = new ApolloServer({ typeDefs, resolvers, middleware: [permissions]});功能:声明式权限控制字段级授权类型安全易于维护Envelopimport { envelop } from '@envelop/core';import { useAuth } from '@envelop/auth';import { useRateLimiter } from '@envelop/rate-limiter';const getEnveloped = envelop({ plugins: [ useAuth({ resolveUserFn: (context) => context.user }), useRateLimiter({ tokenLimit: 100, windowSize: 10000 }) ]});功能:可插拔的插件系统认证和授权速率限制查询复杂度限制7. 文档工具GraphQL Docs# 生成文档npx @graphql-docs/cli generate \ --schema ./schema.graphql \ --output ./docs.md功能:从 Schema 生成文档支持多种格式(Markdown、HTML)自定义模板集成到 CI/CDSpectaQL# 生成交互式文档npx spectaql \ --schema ./schema.graphql \ --output-dir ./docs功能:生成交互式文档网站支持查询测试自定义主题多语言支持8. 微服务工具Apollo Federationconst { ApolloServer } = require('@apollo/server');const { buildSubgraphSchema } = require('@apollo/subgraph');const server = new ApolloServer({ schema: buildSubgraphSchema({ typeDefs, resolvers })});功能:构建联合图分布式 Schema独立部署类型安全的跨服务查询GraphQL Meshimport { createMeshHTTPHandler } from '@graphql-mesh/http';import { loadGraphQLConfig } from '@graphql-mesh/config';const config = await loadGraphQLConfig();const handler = createMeshHTTPHandler(config);// 使用 handlerapp.use('/graphql', handler);功能:整合多个 GraphQL 和 REST API自动生成联合 Schema支持多种数据源无需修改现有 API9. 监控和调试Apollo Tracingconst { ApolloServerPluginUsageReporting } = require('apollo-server-core');const server = new ApolloServer({ typeDefs, resolvers, plugins: [ ApolloServerPluginUsageReporting({ includeRequestContext: true, includeResponseContext: true }) ]});GraphQL Inspector# 安装 Chrome 扩展# 或使用 CLInpx graphql-inspector diff \ --schema ./schema.graphql \ --endpoint https://api.example.com/graphql功能:Schema 变更检测查询分析性能分析安全审计10. 工具选择指南| 场景 | 推荐工具 ||------|----------|| 服务器开发 | Apollo Server, GraphQL Yoga || 客户端开发 | Apollo Client, URQL || 大型应用 | Relay || 类型生成 | GraphQL Code Generator || 测试 | GraphQL Testing Library || 性能优化 | DataLoader || 安全 | GraphQL Shield, Envelop || 文档 | GraphQL Docs, SpectaQL || 微服务 | Apollo Federation, GraphQL Mesh || 监控 | Apollo Studio, GraphQL Inspector |11. 生态系统最佳实践| 实践 | 说明 ||------|------|| 选择合适的工具 | 根据项目需求选择工具 || 保持工具更新 | 定期更新依赖 || 使用类型生成 | 利用代码生成提高开发效率 || 实施监控 | 使用监控工具追踪性能 || 编写测试 | 使用测试工具确保质量 || 优化性能 | 使用性能优化工具 || 重视安全 | 使用安全工具保护 API || 生成文档 | 使用文档工具自动生成文档 || 集成 CI/CD | 将工具集成到开发流程 || 持续学习 | 关注 GraphQL 生态系统的发展 |12. 常见问题及解决方案| 问题 | 解决方案 ||------|----------|| 工具选择困难 | 根据项目规模和团队经验选择 || 工具版本冲突 | 使用包管理器解决依赖冲突 || 学习曲线陡峭 | 从简单工具开始,逐步学习 || 性能问题 | 使用性能工具分析和优化 || 安全漏洞 | 定期更新工具,使用安全工具 |
阅读 0·2月21日 17:00

GraphQL 客户端开发有哪些关键要点

GraphQL 客户端开发指南GraphQL 客户端开发是构建现代前端应用的关键部分。以下是使用 Apollo Client 和其他 GraphQL 客户端的全面指南。1. Apollo Client 配置基本配置import { ApolloClient, InMemoryCache, HttpLink } from '@apollo/client';const httpLink = new HttpLink({ uri: 'https://api.example.com/graphql', credentials: 'include'});const client = new ApolloClient({ link: httpLink, cache: new InMemoryCache(), defaultOptions: { watchQuery: { fetchPolicy: 'cache-and-network', errorPolicy: 'all' }, query: { fetchPolicy: 'network-only', errorPolicy: 'all' }, mutate: { errorPolicy: 'all' } }});带认证的配置import { ApolloClient, InMemoryCache, createHttpLink, ApolloLink } from '@apollo/client';import { setContext } from '@apollo/client/link/context';const httpLink = createHttpLink({ uri: 'https://api.example.com/graphql'});const authLink = setContext((_, { headers }) => { const token = localStorage.getItem('token'); return { headers: { ...headers, authorization: token ? `Bearer ${token}` : '' } };});const client = new ApolloClient({ link: authLink.concat(httpLink), cache: new InMemoryCache()});2. 查询数据使用 useQuery Hookimport { useQuery, gql } from '@apollo/client';const GET_USERS = gql` query GetUsers($limit: Int, $offset: Int) { users(limit: $limit, offset: $offset) { id name email createdAt } }`;function UserList() { const { loading, error, data, fetchMore } = useQuery(GET_USERS, { variables: { limit: 10, offset: 0 }, notifyOnNetworkStatusChange: true }); if (loading) return <div>Loading...</div>; if (error) return <div>Error: {error.message}</div>; return ( <div> {data.users.map(user => ( <div key={user.id}> <h3>{user.name}</h3> <p>{user.email}</p> </div> ))} <button onClick={() => fetchMore({ variables: { offset: data.users.length }, updateQuery: (prev, { fetchMoreResult }) => { if (!fetchMoreResult) return prev; return { users: [...prev.users, ...fetchMoreResult.users] }; } })} > Load More </button> </div> );}使用 lazy queryimport { useLazyQuery, gql } from '@apollo/client';const GET_USER = gql` query GetUser($id: ID!) { user(id: $id) { id name email } }`;function UserSearch() { const [getUser, { loading, error, data }] = useLazyQuery(GET_USER); const [userId, setUserId] = useState(''); return ( <div> <input value={userId} onChange={(e) => setUserId(e.target.value)} placeholder="Enter user ID" /> <button onClick={() => getUser({ variables: { id: userId } })}> Search </button> {loading && <div>Loading...</div>} {error && <div>Error: {error.message}</div>} {data && ( <div> <h3>{data.user.name}</h3> <p>{data.user.email}</p> </div> )} </div> );}3. 修改数据使用 useMutation Hookimport { useMutation, gql } from '@apollo/client';const CREATE_USER = gql` mutation CreateUser($input: CreateUserInput!) { createUser(input: $input) { id name email } }`;function CreateUserForm() { const [createUser, { loading, error }] = useMutation(CREATE_USER, { update(cache, { data: { createUser } }) { cache.modify({ fields: { users(existingUsers = []) { const newUserRef = cache.writeFragment({ data: createUser, fragment: gql` fragment NewUser on User { id name email } ` }); return [...existingUsers, newUserRef]; } } }); } }); const [name, setName] = useState(''); const [email, setEmail] = useState(''); const handleSubmit = (e) => { e.preventDefault(); createUser({ variables: { input: { name, email } } }); }; return ( <form onSubmit={handleSubmit}> <input value={name} onChange={(e) => setName(e.target.value)} placeholder="Name" /> <input value={email} onChange={(e) => setEmail(e.target.value)} placeholder="Email" /> <button type="submit" disabled={loading}> {loading ? 'Creating...' : 'Create User'} </button> {error && <div>Error: {error.message}</div>} </form> );}乐观更新const [updateUser, { loading }] = useMutation(UPDATE_USER, { optimisticResponse: (variables) => ({ updateUser: { __typename: 'User', id: variables.id, name: variables.input.name, email: variables.input.email } }), update(cache, { data: { updateUser } }) { cache.writeFragment({ id: `User:${updateUser.id}`, fragment: gql` fragment UpdateUser on User { name email } `, data: updateUser }); }});4. 缓存管理配置缓存策略const cache = new InMemoryCache({ typePolicies: { Query: { fields: { users: { keyArgs: ['filter', 'sort'], merge(existing = [], incoming) { return [...existing, ...incoming]; } }, read(existing, { args: { offset, limit } }) { return existing && existing.slice(offset, offset + limit); } }, user: { read(_, { args, toReference }) { return toReference({ __typename: 'User', id: args.id }); } } } }, User: { keyFields: ['id', 'email'] } }});手动更新缓存import { useApolloClient } from '@apollo/client';function UpdateUserButton({ userId, newData }) { const client = useApolloClient(); const handleClick = () => { client.writeFragment({ id: `User:${userId}`, fragment: gql` fragment UserFragment on User { name email } `, data: newData }); }; return <button onClick={handleClick}>Update User</button>;}清除缓存function ClearCacheButton() { const client = useApolloClient(); const handleClick = () => { client.clearStore(); }; return <button onClick={handleClick}>Clear Cache</button>;}5. 分页基于偏移的分页const GET_POSTS = gql` query GetPosts($offset: Int, $limit: Int) { posts(offset: $offset, limit: $limit) { id title content author { name } } }`;function PostList() { const { loading, data, fetchMore } = useQuery(GET_POSTS, { variables: { offset: 0, limit: 10 } }); return ( <div> {data?.posts.map(post => ( <div key={post.id}> <h3>{post.title}</h3> <p>{post.content}</p> </div> ))} <button onClick={() => fetchMore({ variables: { offset: data.posts.length }, updateQuery: (prev, { fetchMoreResult }) => { if (!fetchMoreResult) return prev; return { posts: [...prev.posts, ...fetchMoreResult.posts] }; } })} > Load More </button> </div> );}基于游标的分页const GET_POSTS = gql` query GetPosts($after: String, $first: Int) { posts(after: $after, first: $first) { edges { node { id title content } cursor } pageInfo { hasNextPage endCursor } } }`;function PostList() { const { loading, data, fetchMore } = useQuery(GET_POSTS, { variables: { first: 10 } }); return ( <div> {data?.posts.edges.map(({ node }) => ( <div key={node.id}> <h3>{node.title}</h3> <p>{node.content}</p> </div> ))} {data?.posts.pageInfo.hasNextPage && ( <button onClick={() => fetchMore({ variables: { after: data.posts.pageInfo.endCursor }, updateQuery: (prev, { fetchMoreResult }) => { if (!fetchMoreResult) return prev; return { posts: { ...fetchMoreResult.posts, edges: [ ...prev.posts.edges, ...fetchMoreResult.posts.edges ] } }; } })} > Load More </button> )} </div> );}6. 错误处理处理 GraphQL 错误function UserList() { const { loading, error, data } = useQuery(GET_USERS); if (loading) return <div>Loading...</div>; if (error) { if (error.graphQLErrors) { return ( <div> GraphQL Errors: {error.graphQLErrors.map((err, i) => ( <div key={i}>{err.message}</div> ))} </div> ); } if (error.networkError) { return <div>Network Error: {error.networkError.message}</div>; } return <div>Error: {error.message}</div>; } return <div>{/* render data */}</div>;}全局错误处理import { ApolloClient, InMemoryCache, ApolloLink, HttpLink } from '@apollo/client';import { onError } from '@apollo/client/link/error';const errorLink = onError(({ graphQLErrors, networkError, operation, forward }) => { if (graphQLErrors) { graphQLErrors.forEach(({ message, locations, path }) => { console.error( `[GraphQL error]: Message: ${message}, Location: ${locations}, Path: ${path}` ); }); } if (networkError) { console.error(`[Network error]: ${networkError}`); }});const httpLink = new HttpLink({ uri: 'https://api.example.com/graphql'});const client = new ApolloClient({ link: errorLink.concat(httpLink), cache: new InMemoryCache()});7. 订阅使用 useSubscription Hookimport { useSubscription, gql } from '@apollo/client';const MESSAGE_ADDED = gql` subscription OnMessageAdded($roomId: ID!) { messageAdded(roomId: $roomId) { id text author { name } createdAt } }`;function ChatRoom({ roomId }) { const { data, loading, error } = useSubscription(MESSAGE_ADDED, { variables: { roomId } }); if (loading) return <div>Connecting...</div>; if (error) return <div>Error: {error.message}</div>; return ( <div> <h3>New Message:</h3> <p>{data.messageAdded.text}</p> <small>By {data.messageAdded.author.name}</small> </div> );}8. 性能优化使用 persisted queriesimport { createPersistedQueryLink } from '@apollo/client/link/persisted-queries';import { sha256 } from 'crypto-hash';const persistedQueryLink = createPersistedQueryLink({ sha256, useGETForHashedQueries: true});const client = new ApolloClient({ link: persistedQueryLink.concat(httpLink), cache: new InMemoryCache()});批量查询import { useQuery, gql } from '@apollo/client';const GET_MULTIPLE_USERS = gql` query GetMultipleUsers($ids: [ID!]!) { users(ids: $ids) { id name email } }`;function UserList({ userIds }) { const { loading, data } = useQuery(GET_MULTIPLE_USERS, { variables: { ids: userIds } }); if (loading) return <div>Loading...</div>; return ( <div> {data.users.map(user => ( <div key={user.id}>{user.name}</div> ))} </div> );}9. 客户端开发最佳实践| 实践 | 说明 ||------|------|| 使用缓存 | 减少网络请求,提高性能 || 实现乐观更新 | 提升用户体验 || 处理错误 | 提供友好的错误提示 || 使用分页 | 处理大量数据 || 实现加载状态 | 改善用户体验 || 使用订阅 | 实现实时更新 || 优化查询 | 只请求需要的字段 || 使用 persisted queries | 减少网络传输 || 实现重试机制 | 提高可靠性 || 监控性能 | 及时发现性能问题 |10. 常见问题及解决方案| 问题 | 原因 | 解决方案 ||------|------|----------|| 缓存未更新 | 缓存策略配置不当 | 调整缓存策略,手动更新缓存 || 查询慢 | 请求过多数据 | 优化查询,使用字段选择 || 内存占用高 | 缓存数据过多 | 定期清理缓存,限制缓存大小 || 订阅断开 | 网络不稳定 | 实现自动重连机制 || 错误处理不当 | 未正确处理错误 | 实现全局错误处理 |
阅读 0·2月21日 17:00

Kafka 与 RabbitMQ、RocketMQ 有什么区别?

Kafka 与其他消息队列的对比Kafka 作为分布式流处理平台,与传统消息队列(如 RabbitMQ、RocketMQ、ActiveMQ)相比,在设计理念、性能特性和应用场景上都有显著差异。理解这些差异对于技术选型和系统架构设计非常重要。Kafka vs RabbitMQ架构设计Kafka:分布式架构,支持水平扩展基于日志存储,消息持久化到磁盘采用 Pull 模式,Consumer 主动拉取消息无状态 Broker,消息存储在文件系统RabbitMQ:集中式架构,支持集群模式基于内存存储,消息存储在内存或磁盘采用 Push 模式,Broker 主动推送消息有状态 Broker,消息存储在内部数据库性能特性Kafka:高吞吐量:单机可达百万级 TPS低延迟:毫秒级延迟高并发:支持大量并发连接顺序读写:利用磁盘顺序读写优势RabbitMQ:中等吞吐量:单机万级 TPS低延迟:微秒级延迟中等并发:并发连接数有限随机读写:内存访问速度快消息可靠性Kafka:消息持久化到磁盘支持副本机制保证数据不丢失支持消息回溯消息保留时间可配置RabbitMQ:消息可持久化到磁盘支持消息确认机制支持死信队列消息默认不持久化功能特性Kafka:支持消息回溯支持消息压缩支持事务消息支持流处理(Kafka Streams)RabbitMQ:支持消息路由(Exchange、Binding)支持消息优先级支持延迟消息支持消息 TTLKafka vs RocketMQ架构设计Kafka:纯分布式架构无中心化设计基于 ZooKeeper 协调简单的存储模型RocketMQ:分布式架构支持 NameServer 协调支持主从架构复杂的存储模型性能特性Kafka:吞吐量更高延迟稍高批量处理能力强零拷贝技术优化RocketMQ:吞吐量较高延迟较低单条消息处理快事务消息性能好消息可靠性Kafka:副本机制保证可靠性支持同步和异步复制数据持久化到磁盘支持消息回溯RocketMQ:主从同步保证可靠性支持同步双写和异步复制支持消息刷盘策略支持消息重试功能特性Kafka:流处理能力强生态丰富(Kafka Connect、Kafka Streams)社区活跃文档完善RocketMQ:事务消息支持完善支持消息过滤支持定时消息支持消息轨迹Kafka vs ActiveMQ架构设计Kafka:现代分布式架构无状态设计水平扩展能力强存储与计算分离ActiveMQ:传统消息队列架构有状态设计垂直扩展为主存储与计算耦合性能特性Kafka:高吞吐量低延迟高并发顺序读写优化ActiveMQ:中等吞吐量中等延迟低并发传统数据库存储消息可靠性Kafka:副本机制持久化存储消息回溯高可用性ActiveMQ:持久化存储消息确认事务支持主从复制技术选型建议选择 Kafka 的场景大数据场景日志收集实时数据分析流式处理数据管道高吞吐量场景每秒百万级消息批量数据处理大规模数据传输消息回溯需求需要重新消费历史消息需要多订阅者消费需要消息持久化流处理场景实时计算事件驱动架构复杂事件处理选择 RabbitMQ 的场景复杂路由场景需要灵活的消息路由需要消息过滤需要多条件匹配低延迟场景微秒级延迟要求实时性要求高消息量适中企业应用场景企业级消息中间件传统的消息队列需求需要丰富的管理功能选择 RocketMQ 的场景金融场景事务消息需求高可靠性要求消息顺序要求电商场景订单处理库存同步消息轨迹追踪阿里生态使用阿里云服务需要 Spring Cloud 集成需要完善的技术支持选择 ActiveMQ 的场景传统应用JMS 规范要求传统企业应用简单消息队列需求小规模应用消息量不大部署简单维护成本低性能对比总结| 特性 | Kafka | RabbitMQ | RocketMQ | ActiveMQ ||------|-------|----------|----------|----------|| 吞吐量 | 极高 | 中等 | 高 | 低 || 延迟 | 毫秒级 | 微秒级 | 毫秒级 | 中等 || 可靠性 | 高 | 高 | 高 | 中等 || 扩展性 | 极强 | 中等 | 强 | 弱 || 复杂度 | 中等 | 高 | 高 | 中等 || 生态 | 丰富 | 丰富 | 一般 | 一般 |最佳实践根据业务场景选择大数据场景优先选择 Kafka复杂路由场景优先选择 RabbitMQ金融场景优先选择 RocketMQ考虑团队能力选择团队熟悉的技术栈考虑学习和维护成本评估技术支持能力评估长期规划考虑业务增长需求评估技术发展趋势规划技术演进路线通过对比不同消息队列的特性和适用场景,可以做出更合理的技术选型决策。
阅读 0·2月21日 17:00

Kafka 的 Consumer Group Rebalance 机制是什么?

Kafka Consumer Group Rebalance 机制Consumer Group Rebalance 是 Kafka 中一个重要的机制,用于在 Consumer Group 成员变化时重新分配 Partition。理解 Rebalance 机制对于保证 Kafka 消费的稳定性和高可用性至关重要。Rebalance 触发条件1. Consumer 成员变化新 Consumer 加入:新的 Consumer 实例加入 Consumer GroupConsumer 退出:Consumer 实例正常退出或异常退出Consumer 故障:Consumer 宕机或网络中断Consumer 超时:Consumer 超过 session.timeout.ms 未发送心跳2. Partition 数量变化Topic Partition 增加:Topic 的 Partition 数量增加Topic Partition 减少:Topic 的 Partition 数量减少Topic 删除:Topic 被删除3. 订阅变化Consumer 订阅新 Topic:Consumer 开始订阅新的 TopicConsumer 取消订阅:Consumer 取消订阅某个 TopicRebalance 过程1. Rebalance 触发触发条件满足后,Controller 检测到需要 RebalanceController 通知 Group Coordinator 启动 Rebalance2. Join Group 阶段所有 Consumer 向 Group Coordinator 发送 JoinGroup 请求Group Coordinator 选择一个 Consumer 作为 LeaderLeader Consumer 负责制定 Partition 分配方案3. Sync Group 阶段Leader Consumer 将分配方案发送给 Group CoordinatorGroup Coordinator 将分配方案发送给所有 ConsumerConsumer 接收分配方案并开始消费4. 完成阶段Consumer 开始消费分配到的 PartitionRebalance 过程完成Rebalance 策略Range 策略(默认)原理:按照 Partition 的顺序和 Consumer 的顺序进行分配分配规则:将 Partition 按照数字顺序排序将 Consumer 按照名称排序每个 Consumer 分配连续的 Partition 范围示例:Topic: test-topic, Partitions: 0,1,2,3,4,5Consumers: C1, C2, C3分配结果:C1: 0,1C2: 2,3C3: 4,5特点:分配相对均匀可能导致分配不均衡(当 Partition 数不能被 Consumer 数整除时)RoundRobin 策略原理:轮询分配 Partition分配规则:将所有 Topic 的 Partition 合并按照轮询方式分配给 Consumer示例:Topic1: 0,1,2Topic2: 0,1Consumers: C1, C2分配结果:C1: Topic1-0, Topic1-2, Topic2-1C2: Topic1-1, Topic2-0特点:分配更均匀适用于多个 Topic 的情况Sticky 策略原理:在保证分配均匀的前提下,尽量保持原有分配特点:减少 Partition 在 Consumer 之间的移动降低 Rebalance 的影响提高消费的连续性CooperativeSticky 策略原理:增量式 Rebalance,只重新分配受影响的 Partition特点:减少 Stop-the-world 时间提高系统可用性适用于对连续性要求高的场景Rebalance 配置关键参数# 会话超时时间session.timeout.ms=30000# 心跳间隔时间heartbeat.interval.ms=3000# 最大 poll 间隔时间max.poll.interval.ms=300000# Rebalance 超时时间max.poll.records=500参数说明session.timeout.ms:Consumer 超过此时间未发送心跳,将被认为失效heartbeat.interval.ms:Consumer 发送心跳的间隔时间max.poll.interval.ms:Consumer 两次 poll 之间的最大间隔时间max.poll.records:每次 poll 最多返回的消息数Rebalance 问题及解决方案1. Rebalance 频繁触发原因:Consumer 频繁上下线网络不稳定导致心跳超时消费处理时间过长解决方案:# 增加会话超时时间session.timeout.ms=60000# 增加心跳间隔heartbeat.interval.ms=5000# 增加 poll 间隔max.poll.interval.ms=6000002. Rebalance 时间过长原因:Consumer 数量过多Partition 数量过多网络延迟高解决方案:使用 CooperativeSticky 策略减少 Consumer 数量优化网络配置3. Rebalance 导致消息重复消费原因:Rebalance 期间 Consumer 可能重复消费消息Offset 提交不及时解决方案:# 禁用自动提交enable.auto.commit=false# 手动提交 Offsetconsumer.commitSync();4. Rebalance 导致消费中断原因:Rebalance 期间 Consumer 停止消费Stop-the-world 时间过长解决方案:使用 CooperativeSticky 策略减少 Rebalance 触发频率优化 Rebalance 配置最佳实践1. 合理配置参数# 推荐配置session.timeout.ms=30000heartbeat.interval.ms=3000max.poll.interval.ms=300000max.poll.records=5002. 选择合适的 Rebalance 策略一般场景:使用 Range 或 RoundRobin需要连续性:使用 Sticky高可用要求:使用 CooperativeSticky3. 监控 Rebalance# 查看 Consumer Group 状态kafka-consumer-groups --bootstrap-server localhost:9092 \ --describe --group my-group# 查看 Rebalance 日志tail -f /path/to/kafka/logs/server.log | grep Rebalance4. 优化消费逻辑避免长时间处理单条消息使用异步处理提高效率合理设置批量处理大小5. 预防 Rebalance保持 Consumer 稳定运行避免频繁启停 Consumer监控 Consumer 健康状态Rebalance 监控指标RebalanceRatePerSec:每秒 Rebalance 次数RebalanceTotal:Rebalance 总次数FailedRebalanceRate:失败的 Rebalance 比例SuccessfulRebalanceRate:成功的 Rebalance 比例通过合理配置和优化 Rebalance 机制,可以有效减少 Rebalance 对系统的影响,提高 Kafka 消费的稳定性和可用性。
阅读 0·2月21日 17:00

Nginx 如何优化静态资源?有哪些优化策略?

Nginx 如何优化静态资源?有哪些优化策略?Nginx 在提供静态资源方面表现优异,通过合理的配置可以显著提升静态资源的加载速度和用户体验。启用高效文件传输:http { # 启用 sendfile sendfile on; # 启用 tcp_nopush tcp_nopush on; # 启用 tcp_nodelay tcp_nodelay on; server { listen 80; server_name example.com; root /var/www/html; location / { try_files $uri $uri/ =404; } }}Gzip 压缩:http { # 启用 Gzip gzip on; gzip_vary on; gzip_min_length 1024; gzip_comp_level 6; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/rss+xml application/atom+xml image/svg+xml; gzip_disable "msie6"; # 静态资源预压缩 gzip_static on; server { listen 80; server_name example.com; root /var/www/html; location / { try_files $uri $uri/ =404; } }}浏览器缓存:server { listen 80; server_name example.com; root /var/www/html; # 静态资源长期缓存 location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ { expires 1y; add_header Cache-Control "public, immutable"; access_log off; } # HTML 文件短期缓存 location ~* \.html$ { expires 1h; add_header Cache-Control "public, must-revalidate"; } # 不缓存动态内容 location ~* \.php$ { expires off; add_header Cache-Control "no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0"; } location / { try_files $uri $uri/ =404; }}文件缓存:http { # 打开文件缓存 open_file_cache max=100000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on; server { listen 80; server_name example.com; root /var/www/html; location / { try_files $uri $uri/ =404; } }}静态资源分离:server { listen 80; server_name example.com; # 主站点 location / { root /var/www/html; try_files $uri $uri/ =404; } # 静态资源 location /static/ { root /var/www/static; expires 1y; add_header Cache-Control "public, immutable"; access_log off; } # 图片资源 location /images/ { root /var/www/images; expires 1y; add_header Cache-Control "public, immutable"; access_log off; } # 字体文件 location /fonts/ { root /var/www/fonts; expires 1y; add_header Cache-Control "public, immutable"; access_log off; add_header Access-Control-Allow-Origin *; }}CDN 集成:server { listen 80; server_name example.com; root /var/www/html; # 重写静态资源 URL 到 CDN location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2)$ { return 301 https://cdn.example.com$request_uri; } location / { try_files $uri $uri/ =404; }}图片优化:server { listen 80; server_name example.com; root /var/www/html; # 图片缓存 location ~* \.(jpg|jpeg|png|gif|ico|svg|webp)$ { expires 1y; add_header Cache-Control "public, immutable"; access_log off; # WebP 支持 if ($http_accept ~* "webp") { rewrite ^(.+)\.(jpg|png)$ $1.webp last; } } # 图片防盗链 location ~* \.(jpg|jpeg|png|gif)$ { valid_referers none blocked example.com *.example.com; if ($invalid_referer) { return 403; } } location / { try_files $uri $uri/ =404; }}字体文件优化:server { listen 80; server_name example.com; root /var/www/html; # 字体文件 location ~* \.(woff|woff2|ttf|otf|eot)$ { expires 1y; add_header Cache-Control "public, immutable"; access_log off; # CORS 支持 add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Methods "GET, OPTIONS"; add_header Access-Control-Allow-Headers "Origin, Content-Type"; } location / { try_files $uri $uri/ =404; }}静态资源预加载:server { listen 80; server_name example.com; root /var/www/html; location = / { add_header Link "</style.css>; rel=preload; as=style, </script.js>; rel=preload; as=script, </image.jpg>; rel=preload; as=image"; try_files $uri $uri/ =404; } location / { try_files $uri $uri/ =404; }}HTTP/2 推送:server { listen 443 ssl http2; server_name example.com; ssl_certificate /etc/nginx/ssl/example.com.crt; ssl_certificate_key /etc/nginx/ssl/example.com.key; root /var/www/html; # HTTP/2 推送 location = / { http2_push /style.css; http2_push /script.js; http2_push /image.jpg; try_files $uri $uri/ =404; } location / { try_files $uri $uri/ =404; }}静态资源合并:# 使用第三方模块 ngx_http_concat_moduleserver { listen 80; server_name example.com; root /var/www/html; # CSS 合并 location /static/css/ { concat on; concat_types text/css; concat_unique on; concat_max_files 10; } # JS 合并 location /static/js/ { concat on; concat_types application/javascript; concat_unique on; concat_max_files 10; } location / { try_files $uri $uri/ =404; }}完整静态资源优化配置:user nginx;worker_processes auto;worker_rlimit_nofile 65535;events { worker_connections 10240; use epoll; multi_accept on;}http { # 基础优化 sendfile on; tcp_nopush on; tcp_nodelay on; # Gzip 压缩 gzip on; gzip_vary on; gzip_min_length 1024; gzip_comp_level 6; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml+rss application/rss+xml application/atom+xml image/svg+xml; gzip_disable "msie6"; gzip_static on; # 文件缓存 open_file_cache max=100000 inactive=20s; open_file_cache_valid 30s; open_file_cache_min_uses 2; open_file_cache_errors on; # 静态资源长期缓存 map $sent_http_content_type $expires { default off; text/html 1h; text/css 1y; application/javascript 1y; ~image/ 1y; ~font/ 1y; } server { listen 80; server_name example.com; root /var/www/html; index index.html; # 静态资源优化 location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2|ttf|eot)$ { expires $expires; add_header Cache-Control "public, immutable"; access_log off; } # 字体文件 CORS location ~* \.(woff|woff2|ttf|otf|eot)$ { add_header Access-Control-Allow-Origin *; add_header Access-Control-Allow-Methods "GET, OPTIONS"; add_header Access-Control-Allow-Headers "Origin, Content-Type"; } # 图片防盗链 location ~* \.(jpg|jpeg|png|gif)$ { valid_referers none blocked example.com *.example.com; if ($invalid_referer) { return 403; } } # 主路由 location / { try_files $uri $uri/ =404; } # 禁止访问隐藏文件 location ~ /\. { deny all; access_log off; log_not_found off; } }}静态资源优化最佳实践:启用压缩:使用 Gzip 压缩文本资源合理缓存:根据资源类型设置缓存时间文件分离:将静态资源分离到独立域名或 CDN预压缩:使用 gzip_static 预压缩静态文件HTTP/2:启用 HTTP/2 提升加载速度图片优化:使用 WebP 格式,启用图片压缩字体优化:使用 WOFF2 格式,启用 CORS监控性能:使用 Lighthouse 等工具监控性能定期清理:清理未使用的静态资源版本控制:使用文件名哈希实现缓存更新
阅读 0·2月21日 16:58

Nginx 如何处理动态内容?有哪些配置方式?

Nginx 如何处理动态内容?有哪些配置方式?Nginx 本身不处理动态内容,而是通过 FastCGI、uWSGI、SCGI 等协议将请求转发给后端应用服务器处理。FastCGI 配置(PHP):server { listen 80; server_name example.com; root /var/www/html; index index.php index.html; # PHP 文件处理 location ~ \.php$ { try_files $uri =404; fastcgi_pass unix:/var/run/php/php8.0-fpm.sock; fastcgi_index index.php; # FastCGI 参数 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param QUERY_STRING $query_string; fastcgi_param REQUEST_METHOD $request_method; fastcgi_param CONTENT_TYPE $content_type; fastcgi_param CONTENT_LENGTH $content_length; include fastcgi_params; # 超时设置 fastcgi_connect_timeout 60s; fastcgi_send_timeout 60s; fastcgi_read_timeout 60s; # 缓冲区设置 fastcgi_buffer_size 4k; fastcgi_buffers 8 4k; } location / { try_files $uri $uri/ /index.php?$query_string; }}uWSGI 配置(Python):upstream django_backend { server unix:/var/run/uwsgi/app.sock; server 127.0.0.1:8000;}server { listen 80; server_name example.com; root /var/www/html; location / { uwsgi_pass django_backend; include uwsgi_params; # uWSGI 参数 uwsgi_param UWSGI_SCHEME $scheme; uwsgi_param SERVER_NAME $server_name; uwsgi_param REMOTE_ADDR $remote_addr; # 超时设置 uwsgi_connect_timeout 60s; uwsgi_send_timeout 60s; uwsgi_read_timeout 60s; # 缓冲区设置 uwsgi_buffering on; uwsgi_buffer_size 4k; uwsgi_buffers 8 4k; } # 静态文件 location /static/ { alias /var/www/html/static/; expires 1y; add_header Cache-Control "public, immutable"; }}SCGI 配置(Ruby):upstream rails_backend { server unix:/var/run/scgi/rails.sock; server 127.0.0.1:9000;}server { listen 80; server_name example.com; root /var/www/html; location / { scgi_pass rails_backend; include scgi_params; # SCGI 参数 scgi_param SCGI 1; scgi_param SERVER_SOFTWARE nginx; # 超时设置 scgi_connect_timeout 60s; scgi_send_timeout 60s; scgi_read_timeout 60s; }}HTTP 代理配置:upstream nodejs_backend { server 127.0.0.1:3000; server 127.0.0.1:3001;}server { listen 80; server_name example.com; location / { proxy_pass http://nodejs_backend; # 代理头设置 proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Proto $scheme; # 超时设置 proxy_connect_timeout 60s; proxy_send_timeout 60s; proxy_read_timeout 60s; # 缓冲区设置 proxy_buffering on; proxy_buffer_size 4k; proxy_buffers 8 4k; # WebSocket 支持 proxy_http_version 1.1; proxy_set_header Upgrade $http_upgrade; proxy_set_header Connection "upgrade"; }}FastCGI 缓存:# 定义 FastCGI 缓存fastcgi_cache_path /var/cache/nginx/fastcgi levels=1:2 keys_zone=fastcgi_cache:10m max_size=1g inactive=60m;server { listen 80; server_name example.com; root /var/www/html; location ~ \.php$ { try_files $uri =404; # 启用 FastCGI 缓存 fastcgi_cache fastcgi_cache; fastcgi_cache_valid 200 60m; fastcgi_cache_valid 404 1m; fastcgi_cache_key "$scheme$request_method$host$request_uri"; # 缓存跳过条件 fastcgi_cache_bypass $skip_cache; fastcgi_no_cache $skip_cache; fastcgi_pass unix:/var/run/php/php8.0-fpm.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; # 添加缓存状态头 add_header X-Cache-Status $upstream_cache_status; }}动态内容负载均衡:upstream php_backend { least_conn; server 192.168.1.100:9000 weight=3; server 192.168.1.101:9000 weight=2; server 192.168.1.102:9000 weight=1; keepalive 32;}server { listen 80; server_name example.com; root /var/www/html; location ~ \.php$ { fastcgi_pass php_backend; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; }}动态内容压缩:http { # 启用 Gzip 压缩 gzip on; gzip_vary on; gzip_min_length 1024; gzip_comp_level 6; gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml application/rss+xml; server { listen 80; server_name example.com; root /var/www/html; location ~ \.php$ { fastcgi_pass unix:/var/run/php/php8.0-fpm.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; } }}动态内容安全:server { listen 80; server_name example.com; root /var/www/html; # 禁止访问敏感文件 location ~* \.(htaccess|htpasswd|ini|log|sh|sql|bak|old|swp|tmp)$ { deny all; access_log off; } # PHP 文件处理 location ~ \.php$ { # 防止 PHP 文件上传执行 if ($request_filename ~* \.(jpg|jpeg|png|gif|ico)$) { return 403; } fastcgi_pass unix:/var/run/php/php8.0-fpm.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; # 限制请求大小 client_max_body_size 10m; } location / { try_files $uri $uri/ /index.php?$query_string; }}动态内容监控:# 自定义日志格式log_format dynamic '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" ' 'rt=$request_time uct="$upstream_connect_time" ' 'uht="$upstream_header_time" urt="$upstream_response_time" ' 'cache=$upstream_cache_status';server { listen 80; server_name example.com; root /var/www/html; access_log /var/log/nginx/dynamic.log dynamic; location ~ \.php$ { fastcgi_pass unix:/var/run/php/php8.0-fpm.sock; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; # 添加性能头 add_header X-Response-Time $request_time; add_header X-Upstream-Time $upstream_response_time; }}完整动态内容配置示例:user nginx;worker_processes auto;http { # FastCGI 缓存 fastcgi_cache_path /var/cache/nginx/fastcgi levels=1:2 keys_zone=fastcgi_cache:10m max_size=1g inactive=60m; # Gzip 压缩 gzip on; gzip_vary on; gzip_min_length 1024; gzip_comp_level 6; gzip_types text/plain text/css text/xml text/javascript application/json application/javascript application/xml application/rss+xml; # 日志格式 log_format dynamic '$remote_addr - $remote_user [$time_local] ' '"$request" $status $body_bytes_sent ' '"$http_referer" "$http_user_agent" ' 'rt=$request_time uct="$upstream_connect_time" ' 'uht="$upstream_header_time" urt="$upstream_response_time" ' 'cache=$upstream_cache_status'; # PHP 后端 upstream php_backend { least_conn; server 192.168.1.100:9000; server 192.168.1.101:9000; keepalive 32; } server { listen 80; server_name example.com; root /var/www/html; index index.php index.html; access_log /var/log/nginx/example.com.access.log dynamic; error_log /var/log/nginx/example.com.error.log warn; # 禁止访问敏感文件 location ~* \.(htaccess|htpasswd|ini|log|sh|sql|bak|old|swp|tmp)$ { deny all; access_log off; } # 静态资源 location ~* \.(css|js|jpg|jpeg|png|gif|ico|svg|woff|woff2)$ { expires 1y; add_header Cache-Control "public, immutable"; access_log off; } # PHP 文件处理 location ~ \.php$ { try_files $uri =404; # 启用缓存 fastcgi_cache fastcgi_cache; fastcgi_cache_valid 200 60m; fastcgi_cache_valid 404 1m; fastcgi_cache_key "$scheme$request_method$host$request_uri"; fastcgi_pass php_backend; fastcgi_index index.php; fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; include fastcgi_params; # 超时设置 fastcgi_connect_timeout 60s; fastcgi_send_timeout 60s; fastcgi_read_timeout 60s; # 缓冲区设置 fastcgi_buffer_size 4k; fastcgi_buffers 8 4k; # 添加缓存状态头 add_header X-Cache-Status $upstream_cache_status; add_header X-Response-Time $request_time; add_header X-Upstream-Time $upstream_response_time; } # 主路由 location / { try_files $uri $uri/ /index.php?$query_string; } }}动态内容处理最佳实践:使用缓存:启用 FastCGI 缓存减少后端负载负载均衡:使用 upstream 实现负载均衡合理超时:根据业务需求设置超时时间缓冲区优化:调整缓冲区大小提升性能压缩响应:启用 Gzip 压缩减少传输数据安全配置:防止文件上传执行和敏感文件访问监控性能:记录响应时间和缓存命中率静态分离:将静态资源分离到独立路径连接保持:使用 keepalive 减少连接开销错误处理:配置友好的错误页面
阅读 0·2月21日 16:58

RxJS 中的 Marble Testing 是什么?如何使用?

Marble Testing 的概念Marble Testing 是 RxJS 中一种基于字符串的可视化测试方法,它使用特殊的语法来表示 Observable 的时间流和事件。这种方法让异步测试变得直观和易于理解。Marble 语法基本符号| 符号 | 含义 ||------|------|| - | 时间流逝(1帧,约10ms)|| a, b, c | 发出的值 || | | 完成 || # | 错误 || () | 同步发出 || ^ | 订阅点(hot Observable)|| ! | 取消订阅 |示例// 基本示例const source$ = cold('-a-b-c-|');// 含义:10ms后发出a,20ms后发出b,30ms后发出c,40ms后完成// 错误示例const error$ = cold('-a-b-#');// 含义:10ms后发出a,20ms后发出b,30ms后出错// 同步示例const sync$ = cold('(abc|)');// 含义:同步发出a、b、c,然后完成// Hot Observableconst hot$ = hot('^-a-b-c-|');// 含义:从订阅点开始,10ms后发出a,20ms后发出b,30ms后发出c,40ms后完成TestScheduler 的使用基本设置import { TestScheduler } from 'rxjs/testing';describe('My Observable Tests', () => { let testScheduler: TestScheduler; beforeEach(() => { testScheduler = new TestScheduler((actual, expected) => { expect(actual).toEqual(expected); }); });});测试基本操作符import { of } from 'rxjs';import { map, filter } from 'rxjs/operators';it('should map values', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-|'); const expected = '-A-B-C-|'; const result$ = source$.pipe( map(x => x.toUpperCase()) ); expectObservable(result$).toBe(expected, { a: 'a', b: 'b', c: 'c' }); });});it('should filter values', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-d-|'); const expected = '-a-c---|'; const result$ = source$.pipe( filter(x => ['a', 'c'].includes(x)) ); expectObservable(result$).toBe(expected); });});测试时间相关操作符import { of } from 'rxjs';import { delay, debounceTime, throttleTime } from 'rxjs/operators';it('should delay emissions', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-|'); const expected = '---a-b-c-|'; // 延迟30ms const result$ = source$.pipe( delay(30, testScheduler) ); expectObservable(result$).toBe(expected); });});it('should debounce', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a--b---c-|'); const expected = '-----b---c-|'; const result$ = source$.pipe( debounceTime(20, testScheduler) ); expectObservable(result$).toBe(expected); });});it('should throttle', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-d-|'); const expected = '-a---c---|'; const result$ = source$.pipe( throttleTime(30, testScheduler) ); expectObservable(result$).toBe(expected); });});测试组合操作符import { of, merge, concat, combineLatest } from 'rxjs';it('should merge observables', () => { testScheduler.run(({ cold, expectObservable }) => { const source1$ = cold('-a---b-|'); const source2$ = cold('--c-d---|'); const expected = '-a-c-b-d-|'; const result$ = merge(source1$, source2$); expectObservable(result$).toBe(expected); });});it('should concatenate observables', () => { testScheduler.run(({ cold, expectObservable }) => { const source1$ = cold('-a-b-|'); const source2$ = cold('--c-d-|'); const expected = '-a-b--c-d-|'; const result$ = concat(source1$, source2$); expectObservable(result$).toBe(expected); });});it('should combine latest', () => { testScheduler.run(({ cold, expectObservable }) => { const source1$ = cold('-a---b-|'); const source2$ = cold('--c-d---|'); const expected = '----ab-bd-|'; const result$ = combineLatest([source1$, source2$]); expectObservable(result$).toBe(expected); });});测试错误处理import { of, throwError } from 'rxjs';import { catchError, retry } from 'rxjs/operators';it('should catch errors', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-#'); const expected = '-a-b-(d|)'; const result$ = source$.pipe( catchError(() => of('d')) ); expectObservable(result$).toBe(expected); });});it('should retry on error', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-#'); const expected = '-a-a-#'; const result$ = source$.pipe( retry(1) ); expectObservable(result$).toBe(expected); });});测试订阅和取消订阅import { interval } from 'rxjs';import { take } from 'rxjs/operators';it('should handle subscription', () => { testScheduler.run(({ cold, hot, expectObservable, expectSubscriptions }) => { const source$ = cold('-a-b-c-|'); const subs = '^------!'; const result$ = source$.pipe(take(2)); expectObservable(result$).toBe('-a-b-|'); expectSubscriptions(source$.subscriptions).toBe(subs); });});it('should handle unsubscription', () => { testScheduler.run(({ cold, hot, expectObservable, expectSubscriptions }) => { const source$ = cold('-a-b-c-d-|'); const subs = '^---!'; const result$ = source$.pipe(take(2)); expectObservable(result$).toBe('-a-b-|'); expectSubscriptions(source$.subscriptions).toBe(subs); });});实际应用示例1. 测试搜索功能import { of } from 'rxjs';import { debounceTime, distinctUntilChanged, switchMap } from 'rxjs/operators';function search(query: string) { return of(`Results for ${query}`);}it('should search with debounce', () => { testScheduler.run(({ cold, expectObservable }) => { const input$ = cold('-a--b---c-|'); const expected = '-----b---c-|'; const result$ = input$.pipe( debounceTime(20, testScheduler), distinctUntilChanged(), switchMap(query => search(query)) ); expectObservable(result$).toBe(expected); });});2. 测试自动保存import { of } from 'rxjs';import { debounceTime, switchMap } from 'rxjs/operators';function save(data: any) { return of('Saved');}it('should auto-save with debounce', () => { testScheduler.run(({ cold, expectObservable }) => { const changes$ = cold('-a--b---c-|'); const expected = '-----b---c-|'; const result$ = changes$.pipe( debounceTime(20, testScheduler), switchMap(data => save(data)) ); expectObservable(result$).toBe(expected); });});3. 测试轮询功能import { interval } from 'rxjs';import { take, map } from 'rxjs/operators';it('should poll at intervals', () => { testScheduler.run(({ cold, expectObservable }) => { const expected = '-a-b-c-d-e-|'; const result$ = interval(10, testScheduler).pipe( take(5), map(x => String.fromCharCode(97 + x)) ); expectObservable(result$).toBe(expected); });});4. 测试缓存功能import { of } from 'rxjs';import { shareReplay } from 'rxjs/operators';it('should cache values', () => { testScheduler.run(({ cold, expectObservable, expectSubscriptions }) => { const source$ = cold('-a-b-c-|'); const expected = '-a-b-c-|'; const subs = ['^------!', ' ^-!']; const cached$ = source$.pipe(shareReplay(1)); expectObservable(cached$).toBe(expected); expectObservable(cached$).toBe('--c-|'); expectSubscriptions(source$.subscriptions).toBe(subs); });});高级用法1. 测试 Hot Observableit('should handle hot observable', () => { testScheduler.run(({ hot, expectObservable }) => { const source$ = hot('--a--b--c--|'); const sub = '---^--------!'; const expected = '--b--c--|'; const result$ = source$.pipe(take(2)); expectObservable(result$, sub).toBe(expected); });});2. 测试多播import { of } from 'rxjs';import { share, multicast } from 'rxjs/operators';import { Subject } from 'rxjs';it('should multicast correctly', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-|'); const expected = '-a-b-c-|'; const shared$ = source$.pipe(share()); expectObservable(shared$).toBe(expected); expectObservable(shared$).toBe(expected); });});3. 测试自定义操作符import { Observable } from 'rxjs';import { OperatorFunction } from 'rxjs';function customMap<T, R>(project: (value: T) => R): OperatorFunction<T, R> { return (source$) => new Observable(subscriber => { return source$.subscribe({ next: value => { try { subscriber.next(project(value)); } catch (error) { subscriber.error(error); } }, error: error => subscriber.error(error), complete: () => subscriber.complete() }); });}it('should use custom operator', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-|'); const expected = '-A-B-C-|'; const result$ = source$.pipe( customMap(x => x.toUpperCase()) ); expectObservable(result$).toBe(expected); });});最佳实践1. 使用有意义的值// ✅ 好的做法it('should map values', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-|'); const expected = '-A-B-C-|'; const result$ = source$.pipe(map(x => x.toUpperCase())); expectObservable(result$).toBe(expected, { a: 'a', b: 'b', c: 'c', A: 'A', B: 'B', C: 'C' }); });});// ❌ 不好的做法it('should map values', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-a-b-c-|'); const expected = '-d-e-f-|'; const result$ = source$.pipe(map(x => x.toUpperCase())); expectObservable(result$).toBe(expected); });});2. 测试边界情况it('should handle empty observable', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('|'); const expected = '|'; const result$ = source$.pipe(map(x => x.toUpperCase())); expectObservable(result$).toBe(expected); });});it('should handle error observable', () => { testScheduler.run(({ cold, expectObservable }) => { const source$ = cold('-#'); const expected = '-#'; const result$ = source$.pipe(map(x => x.toUpperCase())); expectObservable(result$).toBe(expected); });});3. 使用 expectSubscriptionsit('should subscribe and unsubscribe correctly', () => { testScheduler.run(({ cold, expectObservable, expectSubscriptions }) => { const source$ = cold('-a-b-c-|'); const subs = '^------!'; const result$ = source$.pipe(take(3)); expectObservable(result$).toBe('-a-b-c-|'); expectSubscriptions(source$.subscriptions).toBe(subs); });});总结Marble Testing 是 RxJS 中强大的测试工具,它提供了:可视化测试: 使用字符串表示时间流,直观易懂时间控制: 精确控制异步操作的时序易于维护: 清晰的语法和结构全面覆盖: 可以测试各种操作符和场景掌握 Marble Testing 可以显著提升 RxJS 代码的测试质量和开发效率。
阅读 0·2月21日 16:58

Kafka 为什么能够实现高吞吐量?

Kafka 高吞吐量原理Kafka 之所以能够实现高吞吐量,主要得益于其独特的设计和架构优化。理解这些原理对于性能调优和系统设计非常重要。核心设计原理1. 顺序读写Kafka 采用顺序读写磁盘的方式,这是其高吞吐量的关键因素。优势:顺序读写速度远高于随机读写(可达 100MB/s 以上)减少磁盘磁头移动,降低 I/O 延迟充分利用操作系统的 Page Cache实现:消息以追加方式写入日志文件Consumer 顺序读取日志文件避免随机访问带来的性能损耗2. 零拷贝技术Kafka 使用零拷贝技术减少数据在内核空间和用户空间之间的拷贝次数。传统方式:磁盘 → 内核缓冲区内核缓冲区 → 用户缓冲区用户缓冲区 → Socket 缓冲区Socket 缓冲区 → 网卡零拷贝方式:磁盘 → 内核缓冲区内核缓冲区 → 网卡(直接通过 sendfile 系统调用)优势:减少数据拷贝次数(从 4 次减少到 2 次)减少 CPU 上下文切换提高数据传输效率3. 批量发送Kafka 支持批量发送消息,减少网络请求次数。配置参数:# 批量发送大小batch.size=16384# 批量发送等待时间linger.ms=5优势:减少网络请求次数提高网络利用率降低网络开销4. 页缓存Kafka 充分利用操作系统的页缓存机制。原理:消息写入时先写入页缓存读取时优先从页缓存读取操作系统负责刷盘优势:减少磁盘 I/O提高读取速度利用操作系统的缓存优化5. 分区机制Kafka 通过分区实现并行处理,提高整体吞吐量。优势:不同分区可以并行读写提高并发处理能力分散负载到不同 Broker配置:# Topic 分区数num.partitions=10性能优化配置Producer 配置# 压缩类型compression.type=snappy# 批量发送大小batch.size=32768# 批量发送等待时间linger.ms=10# 缓冲区大小buffer.memory=67108864# 最大请求大小max.request.size=1048576Broker 配置# 网络线程数num.network.threads=8# I/O 线程数num.io.threads=16# 日志刷新间隔log.flush.interval.messages=10000# 日志刷新时间间隔log.flush.interval.ms=1000# 页缓存大小log.dirs=/data/kafka-logsConsumer 配置# 每次拉取最小字节数fetch.min.bytes=1024# 每次拉取最大字节数fetch.max.bytes=52428800# 每次拉取最大等待时间fetch.max.wait.ms=500# 每次拉取消息数max.poll.records=500性能监控指标Producer 指标record-send-rate:消息发送速率record-queue-time-avg:消息在缓冲区平均等待时间request-latency-avg:请求平均延迟batch-size-avg:平均批量大小Broker 指标BytesInPerSec:每秒接收字节数BytesOutPerSec:每秒发送字节数MessagesInPerSec:每秒接收消息数RequestHandlerAvgIdlePercent:请求处理器空闲比例Consumer 指标records-consumed-rate:消息消费速率records-lag-max:最大消费延迟fetch-rate:拉取速率fetch-latency-avg:平均拉取延迟性能调优建议合理设置分区数分区数过多会增加管理开销分区数过少会限制并发能力一般设置为 Broker 数量的倍数优化批量发送根据消息大小调整 batch.size合理设置 linger.ms 平衡延迟和吞吐量监控批量发送效果使用压缩对于文本消息使用 Snappy 或 Gzip对于二进制消息使用 LZ4权衡 CPU 消耗和压缩率监控和调优持续监控性能指标根据监控数据调整配置进行压力测试验证效果硬件优化使用 SSD 提高磁盘性能增加内存提高缓存命中率优化网络配置性能与可靠性的权衡高吞吐量配置可能降低可靠性需要根据业务场景选择合适的配置在关键业务中优先保证可靠性在非关键业务中可以追求更高吞吐量通过理解 Kafka 高吞吐量的原理并进行合理的配置优化,可以在大多数场景下获得优秀的性能表现。
阅读 0·2月21日 16:58