TGINSIGHT SIMILAR POSTS
Изворен канал @pythonotes · Post #183 · 23 ное.
Python + bash Если вам часто требуется запускать shell команды из Python-кода, какой способ вы используете? Самый низкоуровневый это функция os.system(), либо os.popen(). Рекомендованный способ это subprocess.call(). Но это всё еще достаточно неудобно. Советую обратить своё внимание на очень крутую библиотеку sh. Что она умеет? 🔸 удобный синтаксис вызова команд как функций # os import os os.system("tar cvf demo.tar ~/") # subprocess import subprocess subprocess.call(['tar', 'cvf', 'demo.tar', '~/']) # sh import sh sh.tar('cvf', 'demo.tar', "~/") 🔸 простое создание функции-алиаса для длинной команды fn = sh.lsof.bake('-i', '-P', '-n') output = sh.grep(fn(), 'LISTEN') в этом примере также задействован пайпинг 🔸 удобный вызов команд от sudo with sh.contrib.sudo: print(ls("/root")) Такой запрос спросит пароль. Чтобы это работало нужно соответствующим способом настроить юзера. А вот вариант с вводом пароля через код. password = "secret" sudo = sh.sudo.bake("-S", _in=password+"\n") print(sudo.ls("/root")) Это не все фишки. Больше интересных примеров смотрите в документации. Специально для Windows💀 юзеров #libs#linux
Пребарај: #scrapy
@djangoproject · Post #453 · 02.10.2017 г., 20:18
https://medium.com/towards-data-science/using-scrapy-to-build-your-own-dataset-64ea2d7d4673 In short, #Scrapy is a framework built to build web scrapers more easily and relieve the pain of maintaining them. Basically, it allows you to focus on the data extraction using #CSS selectors and choosing XPath expressions and less on the intricate internals of how spiders are supposed to work.
@kankanshu · Post #16374 · 01.12.2025 г., 20:03
慕课网实战课-畅销3年的Python分布式爬虫课程-原版提取 #Python爬虫#分布式爬虫#Scrapy 课程基于真实电商网站，带你从零打造分布式爬虫系统，掌握Scrapy-Redis核心技术与反爬策略，附赠源码及三年持续更新内容，实战性强。 💾 获取资源请点击：👉 点我获取慕课网实战课-畅销3年的Python分布式爬虫课程-原版提取👈
@djangoproject · Post #223 · 05.01.2017 г., 13:36
#scrapy Scrapy is a fast high-level #web crawling and web scraping framework, used to crawl websites and extract structured data from their pages. It can be used for a wide range of purposes, from #data_mining to #monitoring and #automated_testing. https://github.com/scrapy/scrapy
@githubtrending · Post #15387 · 04.01.2026 г., 11:30
#python#crawler#feapder#feaplat#python#scrapy#spider Feapder is a simple, powerful Python web scraping framework (Python 3.6+) with four spider types for different needs, plus breakpoint resuming, monitoring alerts, browser rendering, and massive data deduplication. Install easily via pip (basic, render, or full versions), create a spider with one command, and run it to fetch/parse sites like Baidu. A management system handles deployment/scheduling. This saves you time by making scraping fast, reliable, and scalable without building everything from scratch. https://github.com/Boris-code/feapder
@djangoproject · Post #317 · 28.04.2017 г., 06:15
https://www.dunebook.com/7-best-python-libraries-of-2017/ #TensorFlow #Scrapy #Scikit_learn #OpenAI_universe #zappa #Arrow #Theano
@repo_science · Post #3180 · 12.05.2023 г., 19:53
#webScraping#Python#Scrapy 🐍 Scrapy course - Python web scraping for beginners The Scrapy #Beginners Course will teach you everything you need to learn to start scraping websites at scale using #Python Scrapy. Topics - Creating your first #Scrapy spider - #Crawling through websites & scraping data from each page - Cleaning data with Items & Item Pipelines - Saving data to CSV files, #MySQL & #Postgres#databases - Using fake #user-agents & headers to avoid getting blocked - Using #proxies to scale up your web scraping without getting banned - Deploying your #scraper to the cloud & scheduling it to run periodically 🗣️ Joe Kearney. 🔗Link 📢#youtube ⭐️ Resources ⭐️ Course Resources - Scrapy Docs - Course Guide - Course Github - The Python Scrapy Playbook ----- Main channel: @repo_science Coupons: @freecoupons_reposcience -----
@djangoproject · Post #224 · 07.01.2017 г., 16:53
#AI #automated_testing #automation #asyncio #atexit #button #concurrency #Coroutines #data_mining #dropdownbox #Debian #decorators #django_cms #form #Google #Gym #intelligence #input #lists #machine_learning #map #Metaprogramming #Micro_services #monitoring #Multipart #multi_touch_apps #multiprocessing #Nodes #numerical #OAuth #package #pytest #python #requests #Requests #satellite #scrapy #scikit_learn #SciPy #searching #submit #selectbox #sessions #TensorFlow #text_boxes #text #telegram #Threads #tuples #Universe #urllib #upload
Hashtags
@djangoproject · Post #298 · 17.04.2017 г., 07:42
#AI#Artificial_Intelligence #aiohttp #API #AWS #asyncio #audio #automated_testing #automation #atexit #BeeWare #button #client #concurrency #cron #Coroutine #data_analysis #data_mining #data_processing #database #Deep_Learning #Debian #decorator #dispatch #django #dropdownbox #Docker #event #Firefox #form #freeze #functool #Generator #GeoDjango #Google #GPU #Gym #learn #Image_processing #intelligence #input #IOT #lambda #lists #machine_learning #Magenta #map #Metaprogramming #Micro_services #mind #monitoring #MongoDB #Mozilla #Multipart #multi_touch_apps #multiprocessing #Nodes #NoSQL #numeric_computation #numerical #NumPy #OAuth #object_serialization #OCR #overloading #package #parallel #pipeline #protocols #PostGIS #pyAudioAnalysis #PyInstaller #PySide #PyTorch #pytest #python #Pyvideo_archives #Qt #Redis #random #request #REST #satellite #scrapy #scikit_learn #SciPy #searching #submit #selectbox #Selenium #serialization #server #session #socket #sound #task #TensorFlow #text_boxes #text #test #telegram #Thread #transport #tuples #Universe #Unix #urllib #upload #Web
Hashtags
@djangoproject · Post #425 · 28.08.2017 г., 03:37
#AI#Artificial_Intelligence #aiohttp #AngularJS #API #AWS #asyncio #audio #automated_testing #automation #atexit #BeeWare #button #client #concurrency #Coroutine #cron #curl #data_analysis #data_mining #data_processing #database #Deep_Learning #Debian #decorator #dict #dispatch #django #django_cms #dropdownbox #Docker #event #Firefox #form #Generator #GeoDjango #git #Google #GPU #Gym #learn #Image_processing #intelligence #input #IOT #lambda #learn #lists #machine_learning #Magenta #map #Metaprogramming #Micro_services #mind #monitoring #MongoDB #Mozilla #Multipart #multi_touch_apps #multiprocessing #Nodes #NoSQL #numeric_computation #numerical #NumPy #OAuth #object_serialization #OCR #overloading #package #parallel #pipeline #protocols #PostGIS #pyAudioAnalysis #pycon #Pyflakes #PyInstaller #PySide #PyTorch #pytest #python #Pyvideo_archives #Qt #React #Redis #random #request #REST #satellite #scrapy #scikit_learn #SciPy #searching #submit #selectbox #Selenium #serialization #server #socket #task #telegram #TensorFlow #test #text_boxes #text #tuples #unicode #Universe #Unix #urllib #upload #Web
Hashtags
@djangoproject · Post #513 · 30.11.2017 г., 22:00
#AI#Artificial_Intelligence #AJAX #aiohttp #Anaconda #AngularJS #API #Atom #AWS #asyncio (#Asynchronous) #audio #automated_testing #automation #atexit #BeeWare #Big_Data #bitcoin #blockchain #Bluemix #Brython #button #Celery #client #class #classmethod #concurrency #Coroutine #cron #CSS #curl #data_analysis #data_mining #data_processing #database #Deep_Learning#deep_learning #Debian #decorator #deploy #dict #dispatch #django #django_cms #Django_REST_Framework #dropdownbox #Docker #event #Firefox #Flask #form #functions #Generator #GeoDjango #git #Google #GPU #GUI #Gym #host #HTML #httplib #learn #Image_processing #intelligence #input #Instagram #IOT #iPython #Jupyter #lambda #learn #License #Linux #lists #machine_learning #Magenta #map #Matplotlib #Metaprogramming #Micro_services #Micropython #mind #monitoring #MongoDB #modules #Mozilla #Multipart #multi_touch_apps #multiprocessing #Nodes #NoSQL #numeric_computation #numerical #NumPy #network #neural_network #OAuth #object_serialization #OCR #overloading #package #parallel #pipeline #protocols #PostGIS #pyAudioAnalysis #pycon #Pyflakes #PyInstaller #PyPI #PyQt #PySide #PyTorch #pytest #python #Pyvideo_archives #Qt #Raspberry_Pi #React #Redis #random #request #Regular_Expressions (#re) #REST #RSS #satellite #scikit_learn #SciPy #scrapy #searching #selectbox #Selenium #serialization #server #sessions #single_responsibility_principle #socket #Spark #str #submit #task #telegram #template #TensorFlow #test #text_boxes #text #tuples #unicode #Universe #Unix #unit_test #urllib #upload #uWSGI #Web #WSGI
Hashtags