What are the best Python scripts you've ever written?
I am aputer engineer with 15 years of experience. I have created multiple python scripts (similar to many scripts described already ) for daily usage tasks. However my best python script would be facebook automation. The setup includes a selenium driver on firefox. The script is triggered once every 6 hours on a dedicatedputer. The scripts opens web browser and logs in with my account. Some of thing it can do are listed Parse my full friend list and create an xml with all relevant details. (This is important as later steps take action only on feeds from people in this created xml.) Scroll the feedpage and take actions on individual feeds. By default it will like any profile pic cover pic change. If other people congratulate my friend it can parse thement like the feed andment congratulation message. I am anonymous because most likely it against facebook policies to use this kind of scripts for daily interaction. EDIT 1 This edit section is for people who are interested in knowing how the whole script works. I will try to keep it minimal so that it doesn be too technical. The script has 3 main work areas Navigation Navigate to a webpage scroll the page etc. Action Take some action on specific element based on info collected. ordered-list Navigation Selenium driver gives the direct capability to launch a browser navigate to a scroll down etc. Hence this part is pretty much straight forward. Info collection This is one of the most hard parts. On firefox you can right click any element and inspect it . Inspect Element gives details of what the html code for an element looks like Here is a snapshot of what firefox shows when i inspect a friend name in my friends list. The class of div element is very important. I now know that whenever i will parse an element of this class it will have the details of my friend (name etc ) I first statically find these elements manually and then hardcode them in my script. I can now parse necessary elements and collect the information present in those via selenium. Selenium gives the api to extract each information of an element. For e.g. I can extract the href in above picture and i can save the of my friend. This example also covers first point of my script of how i created xml of all my friends. I need to parse my friends list only once and save it for future use until i add a friend. In a similar way we can parsements count events etc . Action Once we have collected the information we can apply our own programming logic to that information. For e.g if someone hasmented Nice picture we can post a similarment. Selenium provides the api to click on element in a area etc. So for like we simply click on Like element with that specific class. That all folks.
What are some interesting repositories on GitHub that can be used for journalistic purposes?
Good examples would be n Timeline JS a great timeline visualisation tool that can be easily managed via Google Spreadsheets and in any website. TimelineJS s n Open Budget a visualization web app for hierarchical budgets open-budget s n WordPress Post Forking a plugin that adds GitHub logic to WordPress s s n CartoDB Torque a toolkit for mapping time-related big data sets. torque s n simple and lightweight framework for building interactive map applications. s n Luminous Flux the article rethought. lflux s n Real Time Map. real-time-map s n Datawrapper simple yet powerful tool for data visualisations. datawrapper s n Superscrollorama jQuery plugin for creating parallax pages like the now-famous NYT feature uabSnow Fallubb superscrollorama s Make sure to have a look at newly launched Source which is doing pretty much the same as this thread here. They collect code for journalism. s s
What are the most useful gems to use in Rails?
RubyGems were developed to simplify and accelerate the stages of the application creation deployment and library connection. Utilizing this package manager for Ruby saves you time as you get ready-made solutions to almost any task instead of writing the functions from scratch. Each gem contains a particular element of functionality including all related files. Unfortunately they aren structured in any way so in order to find ruby gems it better to use a regular search engine and the required key words (check GitHub s ). Our dedicated development team also actively employs Ruby Gems in the process of software development. Here is the top of the most popular and useful ruby gems according to our experience GeoCoder s . Being able to connect through itself over 4 APIs this Ruby gem implements both the direct and reverse geocoding by IP address geographical coordinates and even real physical addresses (e.g. the address of the street). Bullet s . The most downloaded Ruby gems out there. It was initially created with an intention to boost software performance. It does so by decreasing the total amount of client-server requests. Basically Bullet tracks the N+1 cases of requests and notifies the developer when other tools can be used instead (e.g. cache counter). Pry s . We rmend to simplify the bug fixing procedures for your RoR-based application with the Pry gem which is a more advanced alternative to the standard IRB wrapper. ActiveModelSerializers s (which starts lagging while processingpound documents) and uses caching. Fast JSON API s . Fast JSON API wille in handy when you need fast serialization of software code. It works much faster than Wicked PDF s . This gem is working alongside with wkhtmltopdf s and helps realizing an interaction with the DSL generator. Devise Masquerade s . This Ruby gem helps developing multi user apps. In particular youll be able to test your app from the perspective of users with different levels of access. Devise s . Based on the MVC model the Devise gem can provide secure user authentication and session management. Letter opener s . If you need to create a newsletter mechanism to send notifications to all users that launched your app this gem will help you do that much easier you won need to integrate and configure your own SMTP server. Money Rails s . If you are planning to integrate your app with Ruby Money this gem wille in quite handy. Pundit s . A tool that allows defining different levels of access to the app functionality according to the rights of an authorized user.
How can I read the bar codes present in a PDF file?
How can I read the bar codes present in a PDF file? Venkatesh code I did think some Python Class to suggest you to search for once your asking do not specify any destination found this thread here itself about extracting from PDF and I think it is a good point of start to you Using Python To Extract Text from PDF. question qid 16498 nHope this help.
Is there an easy to use Python library to read a PDF file and extract its text?
the answer is pdfminer as others have said but if the libraries aren working for you it likely because you are expecting too much from them. You need to understand how the pdf file format works as opposed to how format works. Specifically we all expect to be able to use a library to parse some file format for and be able to iterate through the line by line but what if the has no line characters? How would the library know what constitutes a line? Most libraries won try to guess at that and honestly we wouldn want them to because if the line isn represented by a line character then the concept of line isn really part of the (is it?) and we are using the library to extract **. In pdf is laid out meaning that a particular object get displayed at a particular xy position on the page. So what you might think of as 3 lines would actually be 3 objects displayed at (xy) (x y-2) (x y-4) so a extraction library would just pull out the but you have no line data. (IRRC pdfminer hands you String as output just a big String not a (line) iterable it was because PDFMiner didn work for me that I had to study up and learn a bit about pdf to get what I wanted out of the files). The upside is this You finally get a chance to roll your own. Fortunately extracting the out of a pdf is very well defined and simple goal. And fortuanately PDF is a very well documented and very well understood file format so google is going to be very helpful. If pushes to shove the rendering part of the spec is less than 2 pages but you won need to go there. Start here Introduction to PDF s Then read the wikipedia article which is super well written. Then you will have to open the file in editor and study it which won be hard if you are interested only in . Use this as a tool to understand the stream writing operators Adobe Portable Document Format The accepted answer to the following SO tells you what you need to investigate to understand how is encoded within the pdf Programatically rip from a PDF File (by hand) - Missing some Google anything you wish to understand and you will be brought to cool sites like planetpdf where they have great articles. It should take you a day or two to hand write your parser and you will learn a lot in the process about something prettymon. The libraries have to be general so they are going to be limited. (perhaps irrelevant the pdfs I was working with are linearizedsee the ed referenceswhich made studying the in the pdf and mapping to the layout on the screen super simple I didn study an non-linearized files because i didn have to but if it makes things harder there a ton of code out there to linearize a pdf but not a lot out there that can go the otherway)
Other than Open Calais, what are some good tools to extract key words, topics, and tags from a random paragraph of text?
You could try Semantria API ( ) 3 I first tried it using their free trial. I was able to pull names (they call them entities) and themes that reoccurred in my content. By creating queries you can also tag themon topics present in your group of s. They also have a lot of other analysis features that I was interested in like sentiment analysis and categorization.n nOverall I liked their API which was accessible directly within Excel with their plug-in 3 making it easier to use (thank goodness). I don have a lot of experience with other tools but AlchemyAPI and Chatterbox are some others that may offer what you need although they don seem to offer customization of their output or custom sets of categories and tags.
How can I extract an original character from 'CID' in a PDF file?
Source. How can I extract fonts from a PDF as valid font files? s Hope this helps code Using pdftops code One of the most frequently used methods to do this on *nix systems consists of the following steps Convert the PDF to PostScript for example by using XPDF's pdftops code (on Windows code helper program. You may need to convert the .pfa code (ASCII) to a .pfb code (binary) file using the t1utils code and pfa2pfb code . horizontal-rule Using fontforge code Another method is to use the Free font editor FontForge Use the Open Font italic dialogbox used when opening files. Select the PDF file with the font to be extracted. Check the FontForge manual. You may need to follow a few specific steps which are not necessarily straightforward in order to save the extracted font data as a file which is re-usable. horizontal-rule Using mupdf code Next MuPDF . This applicationes with a utility called pdfextract code (on Windows code ) which can extract fonts and s as well as fonts. These include PNG TTF CFF CID etc. The was 412. The fontnames will be like FGETYK+ italic if the font's PDF object number was 966. CFF ( Compact Font Format italic ) files are a recognized format that can be converted to other formats via a variety of converters for use on different operating systems. Again be aware that most of these font files may have only a subset italic of characters and may not represent theplete face. Update (Jul 213) Recent versions of mupdf code have seen an internal reshuffling and renaming of their binaries not just once but several times. The main utility used to be a 'swiss knife'-alike binary called mubusy code (name inspired by busybox?) which more recently was renamed to mutool code . These support the submands info code clean code extract code poster code and show code . Unfortunatey the official documentation for these tools isn't up to date (yet). If you're on a Mac using 'MacPorts' then the utility was renamed in order to avoid name clashes with other utilities using identical names and you may need to use mupdfextract code . To achieve the (roughly) equivalent results with mutool code as its previous tool pdfextract code did just run mubusy extract ... code .* So to extract fonts and images you may need to run one of the followingmandlines c extract # (on Windows) code $ mutool extract filename. code
How can we convert HTML to PDF?
When you save your HTML pages converting it to a PDF format will be a wise decision. This will help you view those HTML pages easily from any device and make it easy to share with others. Now converting these s of HTML pages is only to be done with a converting software. There are various software s to do the conversion but most of them are a bitplicated to use. Rather go for the one that makes the task easier. One such user-friendly software is PDFelement 7 with all the features needed toplete any task with the PDF. Just after installing the software you will have to open the file with the software and then do as follows toplete the conversion. You can also watch the video given below to clearly understand the conversion process from HTML to PDF. s Thanks!
How did you teach yourself data science?
Few points before answering this First I had enough quantitative background to study data science. I am Electrical Engineering grad with signal processing and wirelessmunication as major area. Also I won multiple prizes at National Mathematical Olympiads of my country which practically makes a student with higher mathematical aptitude than most people here. Second I am very curious. I love to learn new stuffs and ask questions. I like to challenge myself both physically and intellectually. Third because of my strong mathematical aptitudes and algorithmic thinking learning new programming languages seems so easy to me. I learnt R programming within 2 months and Python within 3 months. Now let talk on how I start my journey with data science First I started with MOOCs. I took one machine learning from Coursera one linear algebra from MIT one deep learning from Stanford few data science analysis courses from Datacamp etc. Second I played with few projectspleted Kaggle and Drivendatapetitions started exploring Github a lot went through many research papers etc. These helped me to learn more model real life problems and solve them from a data centric perspective. Third I started looking for jobs in the field of machine learning data analysis and data science. Finally I am a data scientist now and working on some amazing problems that might one day solve many problems of country. Thanks!