Unravel

What is Unravel?

Unravel is born to address a problem with Untangle: it’s a good piece of software, but it lacks a precious feature: it cannot import lists of blocked sites in the Web Filter / Web Filter Lite. Well, it CAN import, but only in JSON format.

Migrating to Untangle Web Filter can be a pain: usually you don’t have a list of blocked sites in JSON format!

So I created this small utility: Unravel creates a JSON starting from a simple text file.

Unravel comes already configured to pickup sites from a .txt file (where every line is a site).

So, if you have a .txt file, you can jump directly to the Download section and use it, otherwise you  can adapt it to get the information from a .csv file, but you will need to read my confused and looong explanation.

Explanation

Reading an Untangle export, I discovered that it uses a multidimensional array, where only the ID and the WEBSITE (eventually the description) changes. In simple words, imagine it as a table like this:

 | id | enabled| category | description | flagged | string    | name | javaClass | blocked |
 | 1  |  true  |          | [no descr]  |  true   | google.it |      | blablabla |   true  |
 | 2  |  true  |          | [no descr]  |  true   | yahoo.it  |      | blablabla |   true  |

But in JSON format. (JSON is a way to rappresent variables used in web applications). Althought is a plain text, it isn’t very human friendly: it looks like “flagged”:”true”,”string”:”paginebianche.it”,”name”:””, “javaClass”:”com.untangle.uvm.node.GenericRule”,”blocked”:”1″}

Unravel takes a simple list in TXT or CSV format and create a table like the one Untangle wants, and encode it in JSON format so it can be imported from Untangle web interface with no problems.

Input configuration

Let’s look at the input section of the config file unravel.ini:

[input]
 read_separator=" "
 read_values="@site@,@description@"

This tell Unravel to read every line from a text file: every line is a new site, and a line also can contain the description separed by a space. This configuration will works with an input like:

mylist.txt

www.google.it blocked_by_admin
www.tim.it dont_go_to_tim_it

If you have a configuration file in csv format, like:

mylist.csv

www.google.it;blocked by admin
www.tim.it;please don't visit tim.it

The read_separator value should be set to “;”

Please note that @site@ and @description@ are just bookmarks, you can name whatever you want but you must use strange words to avoid unexpected results. Also you can add much parameters as you like.

Output configuration

[output]
separator=","
columns="id,enabled,category,description,flagged,string,name,javaClass,blocked"
values="@autoinc@,true,,[no description],true,@site@,,com.untangle.uvm.node.GenericRule,true"

Separator is a standard value for the JSON format. You don’t need to change it.

I’ve set the right columns, ready to use with Untangle, so in most of cases you should not change this setting.

Then you must specify a standard row: it will be repeated everytime except for the special value @autoinc@ (it will become a number incrementing from 0) and whatever placemark you want (it will be read from the input file).

Usage example

To import the sites contained in example_sites.txt into Untangle, you must run:

unravel.exe WriteJson example_sites.txt example_output.json

And inport example_output.json into Untangle.

Do you find this software useful? Please support my work with a donation. Even a small amount can motivate me to continue writing free software. Thank you :-)

Download

Unravel runs on all modern version of Windows 32/64 bit and needs the .NET Framework 4.0 because I love to code in PHP even when it’s not a website ;)

unravel.zip - 863.75 kb