Find similar content

Pandas 2.0 将逐步适用 arrow 取代目前的 Numpy 来存储数据。读写性能及处理速度将大为提升。 https://datapythonista.me/blog/pandas-20-and-the-arrow-revolution-part-i #python#data

http://pybit.es/codechallenge11.html Inspired by David Beazley's #Generator Tricks for Systems Programmers we ask you to turn the following unix #pipeline into Python code using generators. To get a bunch of .py files you can use our challenges repo you cloned. Or use a project of your own. Note that in our experience one subprocess is not necessarily one generator, for example 'sort|uniq|sort' can be easily combined into one, as well as 'grep|sed'. See our template if you need guidance.

DigitalOcean accepts criticism and switches its #Hacktoberfest to be opt-in: https://github.com/digitalocean/hacktoberfest/pull/596

http://zetcode.com/python/csv/ Python #CSV tutorial shows how to read and write CSV #data with Python csv module. #learn

http://www.csestack.org/python-libraries-for-data-science/ As per the DIKW Pyramid Model, #Data_Science job revolves around finding the information, knowledge from Raw Data. And it can be bundled into the stack of 4 entities: source of #data manage and store data analyze the data display analyzed output (#visualization, statistics)

Dash, announced this year, is an open source library for building web applications, especially those that make good use of #data visualization, in pure Python. It is built on top of #Flask, #Plotly.js and #React, and provides abstractions that free you from having to learn those frameworks and let you become productive quickly. #Dash is a #Python framework for building analytical web applications. No JavaScript required. https://plot.ly/products/dash/

https://github.com/spotify/luigi Writing batch jobs is generally only one part of processing heaps of data; you also have to string all the jobs together into something resembling a #workflow or a #pipeline. #Luigi, created by Spotify and named for the other plucky plumber made famous by Nintendo, was built to "address all the plumbing typically associated with long-running batch processes." With Luigi, a developer can take several different unrelated data processing tasks — "a Hive query, a Hadoop job in Java, a Spark job in Scala, dumping a table from a database" — and create a workflow that runs them, end to end. The entire description of a job and its dependencies are created as Python modules, not as XML config files or another data format, so it can be integrated into other Python-centric projects. #Machine_learning

http://www.aparat.com/v/4nbc9 This talk gives a quick overview of Python's capabilities as a #data_processing and #machine_learning tool through practical examples: gathering data from the web or a local file, validating/modifying it and finally analyzing it to build models for #classification and #prediction#tasks.

http://www.aparat.com/v/miNUS pycon 2016- Andrew Godwin - Reinventing Django for the #Real_Time Web Django has long been tied to the #request_response pattern, but the upcoming "#channels" project changes this and allows #Django to natively support #WebSockets, running tasks after responses, easily handle #long_polling and more. Come and learn about the design, how we're trying to keep things as Django-like as possible, and how you can use it in your projects.

https://blog.wallaroolabs.com/2017/12/stateful-multi-stream-processing-in-python-with-wallaroo/ #Wallaroo is a high-performance, open-source framework for building distributed stateful applications. In an earlier post, we looked at how Wallaroo scales #distributed_state. In this post, we’re going to see how you can use Wallaroo to implement multiple data processing #tasks performed over the same shared #state. We’ll be implementing an application we’ll call “Market Spread” that keeps track of the latest pricing information by stock while simultaneously using that state to determine whether stock order #requests should be rejected. #pipeline