FAQ

Version

0.1

Date

02-06-2021

Contributors

Roberto Fray da Silva, Xiao-Yang Liu

Description

This document contains the most frequently asked questions related to the FinRL Library, based on questions posted on the slack channels and Github issues.

Section 1 Where to start?

Section 2 What to do when you experience problems?

  • If any questions arise, please follow this sequence of activities (it allows us to focus on the main issues that need to be solved, instead of repeatedly answering the same questions):

    • Check if it is not already answered on this FAQ

    • Check if it is not posted on the Github repo issues:https://github.com/AI4Finance-LLC/FinRL-Library/issues

    • Use the correct slack channel on the AI4Finance slack.

Subsection 3.1 Inputs and datasets

  • Can I use FinRL for crypto?

    not yet. We’re developing this functionality

  • Can I use FinRL for live trading?

    not yet. We’re developing this functionality

  • Can I use FinRL for forex?

    not yet. We’re developing this functionality

  • Can I use FinRL for futures?

    not yet

  • What is the best data source for free daily data?

    Yahoo Finance (through the yfinance library)

  • What is the best data source for minute data?

    Yahoo Finance (only up to last 7 days), through the yfinance library. It is the only option besides scraping (or paying for a service provider)

  • Does FinRL support trading with leverage?

    no, as this is more of an execution strategy related to risk control. You can use it as part of your system, adding the risk control part as a separate component

  • Can a sentiment feature be added to improve the model's performance?

    yes, you can add it. Remember to check on the code that this additional feature is being fed to the model (state)

  • Is there a good free source for market sentiment to use as a feature?

    no, you’ll have to use a paid service or library/code to scrape news and obtain the sentiment from them (normally, using deep learning and NLP)

Subsection 3.2 Code and implementation

  • Does FinRL supports GPU training?

    yes, it does

  • The code works for daily data but gives bad results on intraday frequency.

    yes, because the current parameters are defined for daily data. You’ll have to tune the model for intraday trading

  • Are there different reward functions available?

    not yet, but we’re working on providing different reward functions and an easy way to code your own reward function

  • Can I use a pre-trained model?

    yes, but none is available at the moment. Sometimes in the literature you’ll find this referred to as transfer learning

  • What is the most important hyperparameter to tune on the models?

    each model has its own hyperparameters, but the most important is the total_timesteps (think of it as epochs in a neural network: even if all the other hyperparameters are optimal, with few epochs the model will have a bad performance). The other important hyperparameters, in general, are: learning_rate, batch_size, ent_coef, buffer_size, policy, and reward scaling

  • What are some libraries I could use to better tune the models?

    there are several, such as: ray rllib and optuna. You’ll have to implement them by yourself on the code, as this is not supported yet

  • What DRL models can i use with FinRL?

    all the DRL models on Stable Baselines 3. We tested the following models with success: A2C, A3C, DDPG, PPO, SAC, TD3, TRPO. You can also create your own model, using the OpenAI Gym structure

  • The model is presenting strange results OR is not training.

    Please update to latest version (https://github.com/AI4Finance-LLC/FinRL-Library), check if the hyperparameters used were not outside a normal range (ex: learning rate too high), and run the code again. If you still have problems, please check Section 2 (What to do when you experience problems)

Subsection 3.3 Model evaluation

  • The model did not beat buy and hold (BH) with my data. Is the model or code wrong?

    not exactly. Depending on the period, the asset, the model chosen, and the hyperparameters used, BH may be very difficult to beat (it’s almost never beaten on stocks/periods with low volatility and steady growth). Nevertheless, update the library and its dependencies (the github repo has the most recent version), and check the example notebook for the specific environment type (single, multi, portfolio optimization) to see if the code is running correctly

  • How does backtesting works in the library?

    we use the Pyfolio backtest library from Quantopian ( https://github.com/quantopian/pyfolio ), especially the simple tear sheet and its charts. In general, the most important metrics are: annual returns, cumulative returns, annual volatility, sharpe ratio, calmar ratio, stability, and max drawdown

  • Which metrics should I use for evaluting the model?

    there are several metrics, but we recommend the following, as they are the most used in the market: annual returns, cumulative returns, annual volatility, sharpe ratio, calmar ratio, stability, and max drawdown

  • Which models should I use as a baseline for comparison?

    we recommend using buy and hold (BH), as it’s a strategy that can be followed on any market and tends to provide good results in the long run. You can also compare with other DRL models and trading strategies such as the minimum variance portfolio

Subsection 3.4 Miscellaneous

  • What is the development roadmap for the library?

    this is available on our Github repo https://github.com/AI4Finance-LLC/FinRL-Library

  • How can I contribute to the development?

    participate on the slack channels, check the current issues and the roadmap, and help any way you can (sharing the library with others, testing the library of different markets/models/strategies, contributing with code development, etc)

  • What are some good references before I start using the library?

  • What are some good RL references for people from finance? What are some good finance references for people from ML?

  • What new sota models will be incorporated on FinRL?

    please check our development roadmap at our Github repo: https://github.com/AI4Finance-LLC/FinRL-Library

Section 4 References for diving deep into Deep Reinforcement Learning (DRL)

Subsection 4.1 General resources