Create and Test Custom Task Scripts
IMPORTANT
Before you can create and test custom task scripts for your self-service Tetra Data pipelines (SSPs), you must do the following:
- Initialize the TetraScience Software Development Kit (SDK) 2.0 environment.
- Configure Tetra Data Platform (TDP) dependencies and authentication.
For more information, see SSP Setup and Prerequisites.
To create and test custom task scripts for self-service Tetra Data pipelines (SSPs), do the following.
Create a Custom Task Script
To create a custom task script, see Create and Deploy a Task Script in the “Hello, World!” SSP Example.
NOTE
For more information about task script files and their formatting, see Task Script Files.
Modify Your Custom Task Script Folder Structure for Testing Purposes
To test a custom task script’s functionality locally, you must modify it in the following ways.
Testing Task Script Folder Structure
Modify your local task script folder so that it has the following structure:
sspdemo-taskscript
├── __tests__
│ ├── __init__.py
│ └── test_business_logic.py
├── sspdemo_taskscript
│ ├── __init__.py
│ └── print_hello_world.py
├── README.md
├── requirements.txt
├── config.json
└── main.py
For this example setup, the folder is restructured in the following way:
- The folder name of the task script has dashes. The contents of the folder are the TetraScience specific content (for example,
README.md
,requirements.txt
,config.json
, andmain.py
) - A new subfolder named
sspdemo_taskscript
is created that is a Python package that can be imported for testing. This new subfolder contains the business-relevant functions. It has the same folder name as the task-script, but has underscores instead of dashes (to allow for importing). To be a Python package, it must contain an_init_.py
file, but it can be empty. - A new subfolder named
_tests_
is created that is a Python package that is used for testing when callingpytest
. The Python files in this folder import the relevant functions in thesspdemo_taskscript
Python module. To be a Python package, it must contain an_init_.py
file, but it can be empty. - The
main.py
file now has the functions that are exposed by the**config.json**
file. However, it calls the functions from thesspdemo_taskscript
package, which have been tested.
Add ts-sdk
to the Python Package
ts-sdk
to the Python PackageTo test a task script locally and import the Context API in main.py
, you must add it to the Python package by running the following Python Poetry command on the command line in the top-level task script folder:
poetry add ts-sdk
NOTE
If you're going to re-upload this task script to the TDP, make sure that you modify the
requirements.txt
file by exporting the Python Poetry dependencies. For instructions, see Create a Python Package in the “Hello, World!” SSP Example.
Create a Python Package With Task Script Functions and Modified main.py
File
main.py
FileModify the sspdemo-taskscript/sspdemo_taskscript
subfolders so that they include the functions that you want to use in your task script and test. Then, modify the main.py
file to import the sspdemo_taskscript
package and use the relevant functions it contains.
sspdemo-taskscript/sspdemo_taskscript/print_hello_world.py
from ts_sdk.task.__task_script_runner import Context
# The actual function we are using and testing
def print_hello_world(input: dict, context):
print("Hello World!")
return "Hello World!"
sspdemo-taskscript/main.py
from ts_sdk.task.__task_script_runner import Context
from sspdemo_taskscript.print_hello_world import print_hello_world
# Now our main.py calls the functions from the python package we created
# If there were inputs to pass along, it would also do that. Here we pass
# empty arguments
print_hello_world(input={}, context={})
Create a Test Python File (test_business_logic.py
)
test_business_logic.py
)In this example setup, the print_hello_world
function in the main.py
file calls the print_hello_world
function in the sspdemo_taskscript
Python package. The print_hello_world
function takes in two arguments (input
and context
) and returns a string with the following value: "Hello World!"
To create a test case that demonstrates that the function is working as intended, create a file in a text editor with the following text, and then save it in your _tests_
folder.
`sspdemo-taskscript/tests/test_business_logic.py
from sspdemo_taskscript.main import print_hello_world
def test_hello_world():
output = print_hello_world(input={}, context={})
assert output == "Hello World!"
Test the Setup
In your local command line, navigate to the sspdemo-taskscript
directory. Then, run the following Python Poetry command:
$ poetry run pytest
If you receive a Command not found: pytest
error, try removing and then adding the pytest
package again by running the following commands:
poetry remove pytest
poetry add pytest
NOTE
When you run
pytest,
it will run through all of the tests in your test directory and then provide a summary of which tests passed and failed.
Updated 18 days ago