Market price

How to create a stock price watcher using Ruby

How to create a stock price watcher using Ruby and Nokogiri Gem. The repository containing the complete code used on this guide will be linked at the end of the article. This guide aims to show what you can do using Ruby, you will probably find better / professional ways to do it. The only thing we need from the table is the data from the table. The data is stored inside the rows of the table (tr) and the table. The data in the table (td) is stored in the table with specific keys.

picture
Profile picture of Vinicius Campos Carvalho Hacker Noon

Vinicius Campos Carvalho

Full Stack Developer

For my very first article, I decided to write about web scratching and how I built a stock watcher using Ruby and Nokogiri Gem.

First of all, some prerequisites to follow this guide:

  1. Ruby
  2. HTML / CSS basics

The purpose of this guide is to show what you can do using Ruby, you will likely find better / professional ways to do it.

The repository containing the complete code used on this guide will be linked at the end of the article.

Let’s start

First, set up your environment, create a ruby ​​file that will contain our scrapper code.

In this case, I’ll create a file called stock_watcher.rb and add the following code to it.

#stock_watcher.rb
require 'nokogiri'
require 'open-uri'

This code requires a Ruby Gem called Nokogiri (Help us manage data fetched from website) and the open-uri module to fetch data from the web.

After that we need to set from where we are going to get the data for our scrapper, here you can use any website you want, for this guide i will be using TradingView data.

With the source set now we need to get the page data, Nokogiri has some useful analysis methods to help us, we will use the

Nokogiri::HTML

parser since we are scratching an HTML page.

Add the following code to stock_watcher.rb

#stock_watcher.rb
page = Nokogiri::HTML(URI.open('https://www.tradingview.com/markets/stocks-usa/market-movers-large-cap'))

This can just get the data from the web using URI module and parse it using Nokogiri, it returns a

Nokogiri::HTML::Document

.

With our page object, we have a bunch of useful methods in our hands.

Now we need to use our HTML / CSS acknowledgment to define where on the page we want to get the data from. Chrome Developer Tools are a good friend here.

The page in question is https://www.tradingview.com/markets/stocks-usa/market-movers-large-cap/, this page shows us US STOCK MARKET companies ranked by large cap. This page stores the data on the main table as you can see, what we need to find is a way to identify this table.

Nokogiri offers us a method called “css” which allows us to search for specific CSS rules on our

Nokogiri::HTML::Document

, we can search for specific CSS classes, IDs, etc.

Let’s try to find out what is the class or the ID of the table using the Chrome inspection tool, right click on any item in the table and then inspect, you will find a window similar to that -this :

picture

Scroll down a bit until you find the tag

like this :
picture

As you can see, our table uses the “tv-data-table” class, this will be our target class.

Now that we have our table class, add the following code to stock_watcher.rb:

#stock_watcher.rb
table = page&.css('.tv-data-table');

This code retrieves our page object and finds an HTML element with the “tv-data-table” class assigned to it and returns a

Nokogiri::XML::NodeSet

.

The only thing we need from the table is the data from the table, the data from the table (td) is stored in the rows of the table (tr).

Nokogiri::XML::NodeSet

offers more useful methods that allow us to search for specific keys inside.

Add the following code to stock_watcher.rb:

#stock_watcher.rb
rows = table.search('tr');

This returns another

Nokogiri::XML:NodeSet

with a subset of keys and values ​​of the table that contains ‘tr’.

We are getting closer to our goal.

Now add the following code to stock_wacher.rb:

rows.each do |row|
  stock_content = row.search('td').map { |td| td.text.strip }
  stock_content[0]&.gsub!("nttttttttt", " ")
end

Now we are walking through our XML subset, in this case these are our rows containing the action data. Then we search each row for the data in the table (td) using the same search method, it returns an array with all the data in the iterated row. We need to delete our text because Nokogiri returns a lot of unnecessary chunks of text inside our strings.

Now you can play around with this code a bit, try printing the contents of the stock you got using “puts stock_content” before the end keyword of the last code and see what you got, that should return something something similar to this:

"["MSFT Microsoft Corp.", "182.51", "-2.27%", "-4.23", "Buy", "31.952M", "1384.054B", "31.09", "6.06", "144000.00", "Technology Services"]"

With this data in hand, who defines what needs to be done next is you, there are thousands of possibilities of what could be done with this, I will give a few examples: store data on a file for verification later, store data on a database to manage it later, even viewing the data on another website (check permissions first), create triggers if the value of a stock matches the price you wish. Like I said, there are a lot of possibilities.

Now, to finish this guide, I’ve created a class to handle the data we got from our last piece of code.

First, create another file on the same folder called stock.rb and add the following code:

#stock.rb
class Stock
  attr_reader :name, :last, :change_percentage, :change_value, :rating, :vol, :market_cap
  def initialize(stock_data)
    @name = stock_data[0]
    @last = stock_data[1]
    @change_percentage = stock_data[2]
    @change_value = stock_data[3]
    @rating = stock_data[4]
    @vol = stock_data[5]
    @market_cap = stock_data[6]
  end
end

I’m not going to explain how classes work on Ruby since that’s not the purpose of this guide. But basically this class receives the data we got on the “stack_content” variable and stores it elegantly.

Now replace all your code on stock_watcher.rb file to look like this:

#stock_watcher.rb
require 'nokogiri'
require 'open-uri'
require_relative 'stock'
page = Nokogiri::HTML(URI.open('https://www.tradingview.com/markets/stocks-usa/market-movers-large-cap'))
table = page&.css('.tv-data-table');
rows = table.search('tr');
stock_arr = []
rows.each do |row|
  stock_content = row.search('td').map { |td| td.text.strip }
  stock_content[0]&.gsub!("nttttttttt", " ")
  stock_arr << Stock.new(stock_content) if stock_content.length.positive?
end

It takes our new class file, creates a new array called stock_arr, and stores each new stock object in it.

If you print the first object to stock_arr, you will get something like this:

"#", @last="182.51", @change_percentage="-2.27%", @change_value="-4.23", @rating="Buy", @vol="31.977M", @market_cap="1384.054B">"

As you can see, this is a better way to store our inventory, and you can access its properties using dot notation, like stock.name, stock.last, stock.change_value, etc.

Thanks for reading this guide, hope it helps you somehow.

Please feel free to contact me on any social network listed on HackerNoon.

Useful links:

Profile picture of Vinicius Campos Carvalho Hacker Noon

Key words

Join Hacker Midi

Create your free account to unlock your personalized reading experience.


Source link