Недавно делал быстрый прототип асинхронного приложения в котором требовалось вызывать много синхронного кода. Да, я знаю, что это не лучший дизайн, но нужно было быстрое решение на один процесс и без очередей. Поэтому я выполнял код в потоках.
Выглядело это примерно так:
from fastapi.concurrency import run_in_threadpool
async def execute(data: DataRequest) -> DataResponse:
try:
result = await run_in_threadpool(sync_function, data)
return DataResponse(data=result)
except Exception as e:
return DataResponse(
error=str(e),
success=False,
)
В общем работает нормально. Для всех вызовов под капотом используется общий тредпул, всё работает предсказуемо.
Но потребовалось изменить количество запускаемых в пуле потоков (по умолчанию создается 40 воркеров).
Так как дело происходит с FastAPI, делается это через lifespan используя настройки anyio:
import anyio
@asynccontextmanager
async def lifespan(app: FastAPI):
limiter = anyio.to_thread.current_default_thread_limiter()
limiter.total_tokens = 100
yield
# если вдруг нужно вернуть обратно
limiter.total_tokens = 40
Зачем менять количество воркеров?
- уменьшить, если оперативки мало (один тред занимает ~8мб)
- увеличить чтобы выдержать нагрузку
Если есть предложения получше при тех же вводных - предлагайте😉
#async
http://aiohttp.readthedocs.io/en/stable/web.html#aiohttp-web-websockets
In order to implement a #web_server, first create a #request handler.
A request handler is a coroutine or regular function that accepts a Request instance as its only parameter and returns a Response instance:
#aiohttp#asyncio
from aiohttp import web
async def hello(request):
return web.Response(text="Hello, world")
Next, create an Application instance and register the request handler with the application’s #router on a particular HTTP method and path:
https://github.com/aio-libs/aiohttp/blob/master/docs/web_reference.rst
Server Reference
#asyncio#aiohttp#Request#BaseRequest#Server#client#Router#Resource
#typescript#ai_gateway#gateway#generative_ai#hacktoberfest#langchain#llama_index#llmops#llms#openai#prompt_engineering#router
The AI Gateway by Portkey lets you connect to over 1600 AI models quickly and securely through one simple API, making it easy to integrate any language, vision, or audio AI model in under two minutes. It ensures fast responses with less than 1ms latency, automatic retries, load balancing, and fallback options to keep your AI apps reliable and scalable. It also offers strong security with role-based access, guardrails, and compliance with standards like SOC2 and GDPR. You can save costs with smart caching and optimize usage without changing your code. This helps you build powerful, cost-effective, and secure AI applications faster and with less hassle.
https://github.com/Portkey-AI/gateway
YouTube Issues and Economic Updates
🔧 Users in Russia report ongoing issues with YouTube, marking another decrease in platform traffic, as confirmed by Google.
📊 The Russian Communications Ministry (RKN) plans to acquire data on user attempts to access blocked sites, though it already collects some relevant data.
⚙️ The Ministry of Economic Development aims to increase processing limits to enhance labor market flexibility amid personnel shortages.
📈 Predictions suggest the information security market in 2024 could grow by 30% to reach 593 billion rubles, though other estimates are lower.
📺 Yandex is negotiating with Haier, TCL, and Huawei for the installation of its OS on all their TVs supplied to Russia.
💰 AI search engine Perplexity successfully raised $500 million at a valuation of $9 billion, a significant increase from its earlier valuation of $1 billion.
🇺🇸 In the US, an investigation has been initiated against TP-Link over national security concerns, as they hold 65% of the domestic router market.
#YouTube#RKN#EconomicDevelopment#MarketGrowth#InformationSecurity#Yandex#Perplexity#Funding#TPLink#NationalSecurity#Russia#TechNews#AI#Router#Television#DataPrivacy#UserExperience#TrafficIssues