This article includes all the necessary changes that should be made before starting the server. You should have already created all the required tables in the database using the Tables.sql file.

To assist you in setting up the server and preparing for future work, here are some articles:

• Server set up – python on Ubuntu with cron.
• Server set up – SQL DB part.
• Server set up – SQL requests.
• Market Data with Python and Beautiful Soup.
• DATA scraping with Python scripts (Pandas).

Instruction:

1. After installing all the necessary software, libraries, and tables in the database, BeautifulSoup.py initiates the scraping of quotes at a specific time, as configured in the cron job. This entire process is automated with Cron.
2. It retrieves all the required tickers from the YOUR_FOLDER_PATH/WITH_NECESSARY_TICKER_LIST.txt file, which is stored on your server, and creates a specific file for each ticker in the /YOUR_FOLDER_PATH_FOR_NEW_DATA/ directory on the server.
3. Using Main_template.py and the configured cron job, the data from these files is added to the PostgreSql database table called nyse.nyse_data_new. Please ensure you add the specific request from requests.sql.
4. Subsequently, next SQL script from requests.sql using Main_template.py, automatically adds the necessary data from the nyse.nyse_data_new table to the nyse.nyse_data table, while simultaneously logging an information line in nyse.error_log.
5. Another Python script checks the requests specified in requests.sql and creates a file with missing tickers. The BeautifulSoup.py script is then used to download the missing tickers into the main directory.
6. Finally, the missing tickers are added to the nyse.nyse_data_new table in the database, and the entire process is repeated at the specific time set in the cron job.
7. Lastly, you have the option to set up Telegram messages to receive data from the database. You can use the telegram.py script for this purpose.

The final structure includes the following components:

PostgreSql DB:

Schema: nyse.
Tables: nyse.nyse_data
        nyse.nyse_data_new
        nyse.ticker_list
        nyse.file_path
        nyse.error_log
Functions: nyse.load_csv_files()

Python scripts:

• BeautifulSoup.py: This script is used twice, first for retrieving the main data and then for handling missing tickers.
• Main_template.py: This script performs the following tasks:
  1. Adding new data to the database.
  2. Comparing and creating a file for missing tickers.
  3. Adding the missing data to the main database.
• Telegram.py: This script sends messages to Telegram. You can modify the request to obtain any desired information from the database.”

I hope this revision accurately conveys your intended meaning and improves the clarity of the text.

I am available at all times for you to reach out, and I am more than capable of handling all the necessary procedures on your behalf, for a nominal fee.

• GitHub