Automation – Swap Backgrounders and VizQL dynamically

Tableau V2018.2 introduced Tableau Server Manager (TSM) that is actually a new Tableau server architecture. Tableau does not change its architecture often at all. I do not remember when Tableau had it last time. TSM comes with many awesome features like support for dynamic topology changes,  no dedicated backup primary, installing new server version while current version is still running. This blog talks about the dynamic topology changes.

The problem statement:  Tableau server licenses are expensive. How to get more out of your existing Core server licenses? Or how to get more efficiency out of your HW infrastructure for subscription model?

Contents: Tableau’s Backgrounder handles extract refresh and VizQL handles viz render. Often VizQL has more idle time during night while Backgrounder has more idle time during day. Is it possible to automatically config more  cores as Backgrounders during the night and more VizQL during the day? The dream becomes true with Tableau TSM.

How? Here is the scripts :

set PATH_BIN=”D:\Tableau\Tableau Server\packages\bin.2019.1.xxx”
set PATH_LOGFILE=”xxxx\topology.log”  ##location of the log###
set DATE_TODAY=%DATE:/=-%
set USERNAME=””
set PASSWORD=””
set BG=”4″
set VIZ=”0″
set NODE=”node2″
cd /d %PATH_BIN%

echo %date% %time%: ##### Topology change started ##### >>%PATH_LOGFILE%

echo %date% %time%: Check current topology >> %PATH_LOGFILE%
call tsm topology list-nodes -v -u %USERNAME% -p %PASSWORD% >> %PATH_LOGFILE%

echo %date% %time%: Changing topology to %BG% backgrounders >> %PATH_LOGFILE%
call tsm topology set-process –count %BG% –node %NODE% –process backgrounder -u %USERNAME% -p %PASSWORD% >> %PATH_LOGFILE%

echo %date% %time%: Changing topology to %VIZ% vizql >> %PATH_LOGFILE%
call tsm topology set-process –count %VIZ% –node %NODE% –process vizqlserver -u %USERNAME% -p %PASSWORD% >> %PATH_LOGFILE%

echo %date% %time%: Listing Pending Changes >> %PATH_LOGFILE%
call tsm pending-changes list >> %PATH_LOGFILE%

echo %date% %time%: Apply Pending Changes >> %PATH_LOGFILE%
call tsm pending-changes apply >> %PATH_LOGFILE%

echo %date% %time%: check changes have been applied >> %PATH_LOGFILE%

call tsm topology list-nodes -v >> %PATH_LOGFILE%

echo %date% %time%: ##### script completed ##### >> %PATH_LOGFILE%
:: Finish

Important Notes :

  • It actually works better from V2019.1 onwards.  We found issue with in-fly extract tasks when backgrounder was reduced but issue is fixed in V2019.1.
  • How to decide timing to swap the Backgrounder vs VizQL? All depends on your server usage patten that you can find from historic_events table for user clicks and backgrounder_jobs for number of extracts by hour

Pls comment if you have a better scripts or any other questions

 

6 thoughts on “Automation – Swap Backgrounders and VizQL dynamically”

  1. Hi Mark,

    Your notes state that you found issues with in-fly extract tasks when backgrounder was reduced, in versions before v2019.1.

    Can you elaborate on what the issues were?

    Thanks!

    1. The issue was that when backgrounder was reduced, let’s say 7am. The existing in-flying extracts that started before 7am are supposed to complete successfully before the specific backgrounder becomes unavailable. What it happened for 2018.2 and 2018.3 version was that some of those in-flying extracts got stuck – not regular failures. For regular failed extract, the next one should be able to get started w/o issue. But for the stuck extracts – if it is hourly, it can stuck upto 18 hrs for my server till some kind of internal cleanup job happens. During the 18 hrs, the new hourly extracts can’t be started.
      Good news is that this behavior was fixed in 2019.1

  2. We recently went from 18.1 to 18.3. Tips from Mark here and in previous admin sessions were very helpful and have resulted in a giant reduction of backgrounder delay (from create to start of task). We do see the issue Mark mentioned about tasks going into ‘upside down’ land. We have just made sure we change when nothing is scheduled to run.

  3. Hi Mark,
    What language are you running that script in? Is it Batch / Powershell / Python / Other ?
    Regards,
    Richard

Leave a Reply