PT Magic setup on Ubuntu 17.10

TD;LR

In this post I will explain how to install, configure and run PT Magic (PTM), a free add-on for the Profit Trailer (PT) cryptocurrency trading bot, on a VPS using Ubuntu. As an extra security precaution the PTM GUI is only accessible through a SSH tunnel using PuTTY with port forwarding. This avoids opening additional ports on the VPS to the outside world.

WARNING

To get the full potential out of PT Magic (PTM) you first have to fully understand how Profit Trailer (PT) works. If you don’t understand the basics you are going to over complicate your learning process. You have been warned. Continue at your own risk if you have not taken the time to understand PT.
That said, PTM also provides an impressive GUI that gives you all kind of stats and insights. You can run the GUI without having PTM dynamically change the PT settings by running it in test mode.

What will we cover in this post?

    1. Install .Net Core [server]
    2. Setup Telegram
    3. Setup Putty [client]
    4. Configure Profit Trailer [server]
    5. Install PT Magic [server]
    6. Configure PT Magic [server]
    7. Test Profit Trailer GUI [client]
    8. Setup PM2 [server]
    9. Setup aliases [server]
    10. Full speed ahead [server]
    11. Add another exchange [server]

Prerequisites

To follow this blog post the follow prerequisites must be met:

  • You have basic understanding of Linux and working from the command line.
  • Ubuntu is up and running on a VPS, secured and accessible.
  • Profit Trailer (PT) is up and running.
  • This post assumes you have followed my other posts on setting up Ubuntu and PT. If not you will need to substitute my directories for your own.
  • You have bought an PTF license.
  • This post assumes you have followed my other posts on setting up Ubuntu and PT. If not you will need to substitute my directories for your own.

Okay time to rock & roll. Lets get the show on the road 🙂

Go to index

1. Install .Net Core

PT Magic requires .Net Core to execute. Back in the days getting .Net Core up and running would be a mission. Now it is a breeze. Just execute the following commands.

# Register the Microsoft Product key as trusted
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg sudo mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg

# Set up the desired version host package feed
sudo sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-artful-prod artful main" > /etc/apt/sources.list.d/dotnetdev.list'

# Install .NET Core
sudo apt-get install dotnet-sdk-2.1.3

# Check the version command to prove the installation succeeded
dotnet -version

Go to index

2. Setup Telegram

First we need to setup a Telegram bot so PTM can send Telegram messages.

  • Start a chat with the @BotFather Telegram bot;
  • Enter the command “/newbot”;
  • Enter the name of the bot, e.g. “NidkilPFMagicBot”;
  • Enter the username of the bot, e.g. “nidkil_pf_magic_bot”;
  • Now copy the bot token to use in the next section;
  • Start a chat @chatidbot to get your Telegram chat id;
  • Click the button “Start” at the bottom of the page.

Copy and save the bot token and chat id, as we are going to need it in a later step.

Go to index

3. Setup SSH [client]

Profit Trailer and PT Magic currently do not support two factor authentication (2FA), so I do not feel comfortable making them accessible over the internet. I also do not feel comfortable opening the server up to the outside world, as the server contains a lot sensitive information, e.g. API secret keys. For enhanced security we will use SSH with port forwarding on the client, so that the PT Magic GUI can (only) be accessed through an SSH tunnel. This means that the only port that will be exposed to the outside world is the SSH port, which is more hardened and battle tested than PT Magic GUI.

The following steps are done on the client that will be accessing the server. I have it setup on my Windows laptop, Android phone and Android tablet. For Windows I’m using PuTTY and on Android JuiceSSH. The instruction here are limited to PuTTY.

For these instruction we will be running PT Magic GUI on the port 8100.

PuTTY is an SSH client. It enables you to create profiles to connect to SSH servers.

Note: the assumption is you have Putty up and running with a profile for the server Profit Trailer is running on, so I will only point out the required changes.

2018_01_22_18_03_39_PuTTY_Configuration.png

  • Start PuTTY by double-clicking its executable file or hitting the Windows key and typing “putty” and hitting the “enter” key;
  • Select the profile from the “Load, save or delete a stored session” list.
  • Click the button “Load” to load the profile;
  • Expand the sub-category “SSH” under the category “Connection”;
  • Select the sub-category “Tunnels” under the sub-category “SSH”;
  • Enter the value “8100” in the field “Source port”, enter the value “localhost:8100” in the field “Destination” and click the button “Add”;
  • Return to the category “Session” and enter a name for this profile in the field “Saved Sessions”, e.g. [email protected] or [email protected];
  • Click the button “Save” to save the profile;
  • Click the button “Open” to connect to the server.

Go to index

4. Configure Profit Trailer [server]

In the main PT folder there is the application.properties file. Open it in a text editor.

vim application.properties

Update the following setting (information about the settings can be found here):

trading.logHistory = 9999

This ensures that sales log entries are kept forever. You need to stop and start PT for the change to take effect.

Go to index

5. Install PT Magic [server]

Finally, its time to install PT Magic. Any software that is extracted and run directly should by convention be installed under /opt.

Notes:

  1. For each version of Profit Trailer there will be a version of PT Magic.
  2. You should check the PT Magic release page on Github to get the most recent version and adjust version numbers accordingly.
  3. By convention setup software with the version number in the directory name and have a softlink that points to the active version (in most cases the latest version).
# Create directory for PT Magic
sudo mkdir -p /opt/pt-magic/ptm-binance-v1.0.2

# Optional: create group and add user
sudo adduser nidkil
sudo groupadd profittrailer
sudo useradd -G profittrailer nidkil
# Ensure that the group ownership is inherited by new files and folders created 
sudo chmod -R g+s /opt/pt-magic 

# Change rights so only our user and a specific group can access and execute PT Magic
sudo chmod -R 770 /opt/pt-magic 

# Change ownership to our user and specific group
sudo chown -R nidkil:profittrailer /opt/pt-magic

# Create a soft symbolic link to the current version, comes in handy for aliases
ln -s /opt/pt-magic/ptm-binance-v1.0.2 /opt/pt-magic/ptm-binance-cur

# Change to the PT Magic directory
cd pt-magic/ptm-binance-cur

# Download the latest release of Profit Trailer from GitHub
wget https://github.com/Legedric/ptmagic/releases/download/1.0.2/PTMagic.1.0.2.zip

# Unzip the file
unzip *.zip

# Move the files to the main directory
mv PTMagic\ 1.0.2/* .
mv PTMagic/* .

# Remove the zip
rm *.zip

# Remove the empty directory
rm -rf PTMagic\ 1.0.2 PTMagic

# Copy default configuration files including Monitor directory
cp -r ./_default\ settings/* .

# Copy current Profit Trailer configuration to use as default settings for PT Magic
cp /opt/profit-trailer/pt-binance-cur/trading/* ./_presets/Default/.

Go to index

6. Configure PT Magic [server]

Now the software is installed we need to change a few settings for PTM to work in our environement. We will just do the bare minimum configuration to get PTM running. Fine tuning will be a separate post. In the main PTM folder there is a files settings.general.json. Open it in a text editor.

vim settings.general.json

Update the following settings (information about the settings can be found here):

"ProfitTrailerPath": "[Fully qualified path to the Profit Trailer directory]"
"Exchange": "Binance"
"InstanceName": "PTM Binance"
"Port": 8100
"IsEnabled": true
"BotToken": "[Bot token provided by BotFather in a previous section]"
"ChatId": [Your chat id provided by chatidbot in a previous section]
"SilentMode": true

I have provided the Telegram setup information here, but I must admit that I do not think it (currently) has much added value. It constantly spams you with updates of the market. What gets pretty annoying quickly.

WARNING
Once you start PT Magic it will start adjusting PT settings immediately.

One setting you might want to consider turning on initially is test mode.

"testMode" = true

This will give you time to review the settings of PTM and enjoy the impressive GUI with different stats before having it change your PT settings automatically.

In the main folder you will find the settings.analyzer.json file. This is were the magic happens. This is functionality that is comparable with Profit Trailer Feeder (PTF). These settings controle how the PT settings are dynamically adjusted based on the market settings. Please check out the PT Magic wiki and the file itself to see how this works.

Okay, one more setting to go and we can fire up PTM. Open the monitor configuration file in your text editor.

vim ./Monitor/appsettings.json

We need to let the Monitor know where PT Magic is installed. See here for more information.

"PTMagicBasePath": "[Path to the PT Magic main directory]"

Time to fire up PT Magic and Monitor. Execute the following commands from two separate shells.

dotnet PTMagic.dll

and

dotnet ./Monitor/Monitor.dll

If everything is configured correctly PT Magic and Monitor should startup and you will see logging information showing they are running. If you have enabled Telegram then you should start receiving messages as soon as PT Magic has done its initial data load and analysis, which took about 8 minutes on my server.

Go to index

7. Test Profit Trailer GUI [client]

Now PT Magic and Monitor are up and running lets fire up the browser to access the GUI. Open a browser on the client and enter the following URL.

http://localhost:8100

Impresive GUI, right?

Go to index

8. Configure PM2 [server]

PM2 is a production process manager for applications with a built-in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks. In layman terms, it enables you to start a program and have it continue running in the background even when you logout of the server. We will use pm2 to run Profit Trailer.

Quick overview PM2 commands (see PM2 documentation for complete list)

# Start, daemonize and auto-restart application
pm2 start [/path/to/application]

# List all processes started with PM2
pm2 list

# Display memory and cpu usage of each app
pm2 monit

# Show all information about application
pm2 show [app-name]

# Display logs of all apps
pm2 logs

# Display logs for a specific app
pm2 logs [app-name]

# Start application and name it "api"
pm2 start app.js --name="api"

# Start app and pass option "-a 34" as argument
pm2 start app.js -- -a 34

# Reset all counters
pm2 reset [app-name]

# Stop all apps
pm2 stop all

# Stop process with id 0
pm2 stop 0

# Kill and delete all apps
pm2 delete all

# Delete app with id 0
pm2 delete 0

First lets create two PM2 start scripts to start PT Magic and Monitor. Both scripts will be placed in the main PT Magic directory. Create the pm2-PTMagic.json file to start PT Magic with PM2.

vim pm2-PTMagic.json

PM2 requires an unique application name for every program it starts. Add the following contents to the file and ensure the name is unique:

{
  "apps": [{
   "name": "ptm-binance",
   "cwd": ".",
   "script": "PTMagic.dll",
   "node_args": [],
   "log_date_format": "YYYY-MM-DD HH:mm Z",
   "exec_interpreter": "dotnet",
   "exec_mode": "fork",
   "autorestart": false
  }]
}

This ensures that the PM2 process has an unique and descriptive name. Save and close the file.

Create the pm2-PTM-Monitor.json file to start Monitor with PM2.

vim pm2-PTM-Monitor.json

Add the following contents to the file and ensure the name is unique:

{
  "apps": [{
   "name": "ptm-mon-binance",
   "cwd": ".",
   "script": "Monitor/Monitor.dll",
   "node_args": [],
   "log_date_format": "YYYY-MM-DD HH:mm Z",
   "exec_interpreter": "dotnet",
   "exec_mode": "fork",
   "autorestart": false
  }]
}

Save and close the file. Now it is time to start PT Magic and Monitor with PM2. Shutdown PT Magic and Monitor in the console by pressing the keys Ctrl+C. Use the following command in the main PTM directory to start PT Magic and Monitor using PM2.

pm2 start pm2-PTMagic.json pm2-PTM-Monitor.json

PM2 starts Profit Trailer in the background and shows a list of processes running in PM2. Two of them should be called ptm-binance and ptm-mon-binance. Cool, on to the last step.

Go to index

9. Setup aliases [aliases]

Lets add some aliases for convenience. Aliases are custom commands you can issue from the command line, that execute more complex commands. Open the alias file.

vim ~/.bash_aliases

This file is automatically loaded by ~/.bashrc. On Ubuntu 11.04 and later, it’s enabled by default. Add the following aliases.

alias ptmbincd="cd /opt/pt-magic/ptm-binance-cur/"
alias ptmbinstart="pm2 start /opt/pt-magic/ptm-binance-cur/pm2-PTMagic.json"
alias ptmbinstop="pm2 stop ptm-binance"
alias ptmmbinstart="pm2 start /opt/pt-magic/ptm-binance-cur/pm2-PTM-Monitor.json"
alias ptmmbinstop="pm2 stop ptm-mon-binance"
alias ptmbbinstart="pm2 start ptm-binance ptm-mon-binance"
alias ptmbbinstop="pm2 stop ptm-binance ptm-mon-binance"

The alias commands will be available on any new terminal. To have the alias commands available in the current terminal you need to run one of the aliases you just created.

brl

If you want to list the defined aliases run the following command.

alias

Go to index

10. Full speed ahead [server]

If you are comfortable running the PT Magic and understand the dynamic setings then it is time stop test mode and let it do its magic.

In the main PTM folder open the settings.general.json file.

vim settings.general.json

Update the following setting:

"testMode" = false

That’s all.

Go to index

11. Add another exchange [server]

As a bonus we will add Bittrex. Poloniex is not yet supported. The process is pretty straight forward. Just copy the existing PTM Binance installation and change the exchange specific settings.

Lets start with copying the existing Binance installation.

# Copy the existing Binance PT Magic installation
cp -r /opt/pt-magic/ptm-binance-v1.0.2 /opt/pt-magic/ptm-bittrex-v1.0.2

# Create a soft symbolic link to the current version, comes in handy for aliases
ln -s /opt/pt-magic/ptm-bittrex-v1.0.2 /opt/pt-magic/ptm-bittrex-cur

# Change to Profit Trailer directory
cd /opt/pt-magic/ptm-bittrex-cur

# Clear Binance settings
rm ./_presets/Default/*

# Copy current Profit Trailer configuration to use as default settings for PT Magic
cp /opt/profit-trailer/pt-bittrex-cur/trading/* ./_presets/Default/.
# Clear the log and data directories
rm ./_logs/* ./Monitor/_logs/* 
rm -rf ./_data/*

Now we need to update the exchange specific settings. Open the settings.general.json file.

vim settings.general.json

Change the following settings.

"ProfitTrailerPath": "[Fully qualified path to the Profit Trailer directory]"
"Exchange": "Bittrex"
"InstanceName": "PTM Bittrex" 
"Port": 8101

Open the appsettings.json file.

vim ./Monitor/appsettings.json

Change the following settings.

"PTMagicBasePath": "[Path to the PT Magic Bittrex main directory]"

Now we need to update the PM2 start files so that the correct exchange name is displayed. Open the pm2-PTMagic.json file.

vim pm2-PTMagic.json

PM2 requires an unique application name for every program it starts. Change the following setting.

"name": "ptm-bittrex"

Save and close the file. Open the pm2-PTM-Monitor.json file.

vim pm2-PTM-Monitor.json

Change the following setting.

"name": "ptm-monitor-bittrex"

Now it is time to fire-up the new installation. Execute the following commands from the main directory of PT Magic Bittrex .

pm2 start pm2-PTMagic.json pm2-PTM-Monitor.json

Finally we need to update the PuTTY profile, so that the browser can access the new installation through the SSH tunnel.

  • Start PuTTY by double-clicking its executable file or hitting the Windows key and typing “putty” and hitting the “enter” key;
  • Select the profile from the “Load, save or delete a stored session” list.
  • Click the button “Load” to load the profile;
  • Expand the sub-category “SSH” under the category “Connection”;
  • Select the sub-category “Tunnels” under the sub-category “SSH”;
  • Enter the value “8101” in the field “Source port”, enter the value “localhost:8101” in the field “Destination” and click the button “Add”;
  • Return to the category “Session” and enter a name for this profile in the field “Saved Sessions”, e.g. [email protected] or [email protected];
  • Click the button “Save” to save the profile;
  • Click the button “Open” to connect to the server.

Now open a browser on the client and enter the following URL.

http://localhost:8103

Now lets add some aliases for convenience. Open the alias file.

vim ~/.bash_aliases

Add the following aliases.

alias ptmbitcd="cd /opt/pt-magic/ptm-bittrex-cur/"
alias ptmbitstart="pm2 start /opt/pt-magic/ptm-bittrex-cur/pm2-PTMagic.json" 
alias ptmbitstop="pm2 stop ptm-bittrex"
alias ptmmbitstart="pm2 start /opt/pt-magic/ptm-bittrex-cur/pm2-PTM-Monitor.json"
alias ptmmbitstop="pm2 stop ptm-mon-bittrex"
alias ptmbbitstart="pm2 start ptm-bittrex ptm-mon-bittrex"
alias ptmbbitstop="pm2 stop ptm-bittrex ptm-mon-bittrex"

The alias commands will be available on any new terminal. To have the alias commands available in the current terminal you need to run the following command.

brl

Pretty straight forward, right?

Go to index

Next steps

So whats next?

  • I will be getting to understand PT Magic and the dynamic settings and writing more on this subject in the future.

Summary

This is a basic introduction to installing and configuring PT Magic. In the future I will be posting more blogs about Profit Trailer and PT Magic. Stay tuned!

Related posts

If you liked this post, you might be interested in the following posts.

Go to top of page

Advertisements

19 thoughts on “PT Magic setup on Ubuntu 17.10

  1. Hi Buddy,

    PTMagic nowadays has a feature to password protect your dashboard. Is there any way you can release an update on how I can make the dashboard public (like TP) without having to use Putty with portforwarding? I like to be able to view from my mobile phone as well, but kind of hard that way.

    Regarding pm2, I already have PT running with pm2, and I like it. When I tried to launch PT from another folder using pm2, for some reason it keeps launching the json file in the path you have used in your tutorial. Can I use step 10 in this tutorial? http://nidkil.me/2018/01/22/profittrailer-setup-on-ubuntu-17-10/
    It’s the same exchange, I just want them to have different ID’s (one for testing one which I want to run to have it trading). I understand, both wont be running together (port number conflict).
    Now I want to have PTMagic in the same pm2 list too (lets say: TP production id 0, TP test id 1, PTMagic id 2). How would I achieve that?

    I also have a few questions regarding the setup of it. I have a few questions regarding PTMagic. I know it’s not a product you own, but I love the tutorials you have written. Maybe you can help me out here. I now have a config set up in PT, and I have a small wallet so I have changed the strategy accordingly. When I implement PTMagic, can I set those global rules somewhere? Rules I am talking about are rules such as: ALL_max_trading_pairs, ALL_min_buy_price etc..

    I see you live in the netherlands, so do I. Is there any way I can reach out to you through lets say, Telegram? I did try to look you up there on Telegram, found this hit but then started thinking. How unique is nidkil, maybe there are more as it only shows max 3 search results in telegram (while there could be more). 2 bots at the top and than a “nidkil” which could be you…maybe? 🙂

  2. When I try to register the Microsoft product key as trusted (first command in your tutorial), I get an error back: (23) Failed writing body

  3. Hi.

    You can access the server through SSH from your phone. I do it. That said, yes you can make the website available over internet. But as I state in my blog post, I wouldn’t as the UI hasn’t been battle tested. If I see how they have implemented the password dialog, I do not feel comfortable exposing the UI through internet.

    The reason PM2 is launching the same version probably has to do with the unique id not being unique. Checkout the “name” field in the pm2-PTMagic.json and pm2-PTM-Monitor.json files and make sure it is unique. So you could name them something like ptm-1/ptm-mon-1 and ptm-2/ptm-mon-2. By the way, both versions can run at the same time as long as you have configured different ports. See the “port” field in the settings.general.json file.

    I don’t mind answering some questions, but it is probably better to post them in our Discord channel (https://discord.gg/VanEVCW). As others can profit from the information shared.

  4. Great article! Would you allow us to link to this blog/article in our PTM Wiki?
    Regards,
    TwixRaider

  5. thanks a lot of the GUIDE. Some typos:
    1.
    dotnet -version
    –> dotnet –version
    5.
    ln -s /opt/pt-magic/ptm-binance-v1.0.2 pt-magic/ptm-binance-cur
    –> ln -s /opt/pt-magic/ptm-binance-v1.0.2 /opt/pt-magic/ptm-binance-cur

  6. Pingback: PT Magic management scripts | Adventures of a space monkey

  7. Hi, it depends on your VPS. I have a VPS that is running 6 instances of PT + PTM against 3 exchanges… So if you have enough CPU and memory it is possible.

  8. @Jarnsaxa: I have updated step 5 with instructions how to create a user, group and add the user to the group. You can use the same username and group name as I have or create your own.

  9. Pingback: Profit Trailer management scripts | Adventures of a space monkey

  10. on settings.general.json if OpenBrowserOnStart= true PTMagic Monitor fails to start so set:
    “OpenBrowserOnStart”: false,
    latest dotnet is 2.1.4
    # Install .NET Core
    sudo apt-get install dotnet-sdk-2.1.4

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s