Speedy Web Automation with warp
Warwalrux Awesome Rest Parser
warp
is a tool born of necessity… namely the necessity of having to do things, but not wanting to do the clicky bits to get them done. So I wrote a script that programmatically make rest calls and daisy chain the output together in order to automate larger web processes between multiple disparate sources. In a nutshell, I wanted to be able to query puppetdb with rest and make a webpage, but also update jira tickets based on fancy shmancy confluence-fu (and not the other way around), and show all of my GitHub PRs in a nice compact embedded HTML blurb.
What was crated from that necessity was warp
or “Warwalrux’ Awesome Rest Parser”.
the script draws inspiration from Ansible playbooks in the form of yaml jobfiles which control the scope and sequence of the jobs to be run by warp
.
Basic Example:
warp is a jobfile parser and jinja interpreter for a requests.session backend. This allows users to define complex processes wherein data is manipulated and passed between disparate web services linking them across a single session using ansible-playbook inspired yaml files.
example.yml
is one such job file. When run, warp will read the jobfile, establish session details, interpret the tasks and then run, in sequence, each of the defined tasks.
Getting Warp
Warp currently lives here and can be run straight from the repository after installing the contents of requirements.txt
.
Job Sessions
Warp uses Job Sessions ( an alias to requests.sessions) to manage certificates, auth, and other connection settings for a series of REST calls. The script creates a session (authenticated or not) in order to run the tasks evaluated therein.
A Job must have:
stub
which serves as the base to which taskuri
are joined with a ‘/’.tasks
which is a list of tasks to be run in the session.
A Job may have:
urlencode
// set data to be urlencoded as the default payload formatauth
username
password
verbose
// verbosity booleanvars
headers
Basic Job Example
Authentication
If the authentication
key is present in the job but no username
or pasword
values are found, the script will interactively prompt for values.
The script persists in using these as the default credentials for the entire job.
Certificates
If the certs
key is present in the job, the script will attempt to add the following:
ca_bundle
// ca bundle certificateclient
// client certifcicatekey
// public key
The script persist in using these settings for the entire job.
Job Options
Currently there are two job options available:
verbose
// turn on friendly output and show stepsurlencode
// url encode the payload instead of the default json.
Tasks
a task must have:
uri
// The resource to be called for the taskname
// The friendly display name for the taskaction
// The action statement for the task
if any of there are not present, the script will bail.
a task may have:
loop
// iteration statementurlencode
// urlencode payload data for this task.stub_override
// a URL that will override the job stub ad hoc.data
// YAML formatted request payloadoutput
// an output statement
Example Task
Loops
If the loop key is found in the task object the script will construct items via jinja interpretation. The output will be iterated through and can be referenced by iter_name
This will allow you to use {{ ticket }} as a valid variable for a task
Actions
Task actions are at the core of warp
functionality. All tasks must have valid action statements. Currently only two types of task are supported:
dump
req
dump
Tasks
a dump task lets a user use jinja templating to manipulate a stored variable. When used in conjunction with an output
statement this becomes fairly powerful.
req
tasks
req
tasks are the bread and butter of warp
. They represent the basic REST tasks:
get
put
delete
post
When run the output may be stored or manipulated with an output statement and used in another task or to generate a report.
Output
Output is the directive for the output of the current task. A valid output directive must contain both a write_to
and a content
attribute.
There are three options for redirecting output with write_to
at this time:
file
screen
var
content
may be any one of:
template
// directive to use a template.- additionally a valid jinja2
template_file
ortemplate_str
must be provided.
- additionally a valid jinja2
raw
// an alias for output.contentcontent
// response.contentheaders
// response.headersstatus_code
// response.status_code
If no content
directive is provided the default is response.content
screen
example
Printing stored variable a Jinja Template string
Printing stored variable with a Jinja Template file
Printing task output
file
example
Values will be written to a file with name output[“name”]. If no name is provided the script will interactively prompt for one. This method is useful for reporting
var
example
Values stored as variables can be used in later steps with jinja templating
Task Options
Currently there are two job options available:
urlencode
// url encode the payload instead of the default json.