I have added a new interface for the DeepTrainer library that makes the algorithms accessible through a web interface using queries and which makes it possible to deploy the library in the cloud. The service implements Open Authorization (OAuth) using bearer tokens. Here are the interface functions it implements at the moment:
User account registration:
POST: http://localhost:9000/api/account/register
Login token retrieval for session:
POST: http://localhost:9000/token
Neural network:
GET: http://localhost:9000/api/neuralnetworksetup/parameters/
POST: http://localhost:9000/api/neuralnetworksetup/parameters/
GET: http://localhost:9000/api/neuralnetwork/learn/{iterations}
GET: http://localhost:9000/api/neuralnetwork/state
POST: http://localhost:9000/api/neuralnetwork/state
The application uses an SQL Express database at the moment, but whatever type of database you have at the moment you can use it. Just copy your connection string to the App.Config file, and it is ready to use it as a user registration and login database.
First you have to call the account/register route, with a newly registered user’s details in the body:
[code lang=”java”]
{
"userName": "yourusername",
"password": "yourpassword",
"confirmPassword": "yourpassword"
}
[/code]
Then you have to log in by querying your access token with the above token route. Here you have to add two items in the header:
Accept = application/json
Content-Type = application/x-www-form-urlencoded
Then add these tokens to the body in x-www-form-urlencoded format:
grant_type = password
username = yourusername
password = yourpassword
This query should return you a bearer token like this:
[code lang=”java”]
{
"access_token": "AQAAANCMnd8BFdERjHoAwE_Cl-sBAAAAfiYYyFXhM0ye8EuZPUPL5QAAAAACAAAAAAAQZgAAAAEAACAAAAD5pi7XEc_4Gs5LJ7ge6W9vu0Yf-k4-08nYPrUqUkm4TAAAAAAOgAAAAAIAACAAAABSJdwlubrAFHEL7vMFEWOf0wrD3gsymWwo3qbi_X5o0YAAAAAJzTnD2HHYj_yDycCoDj_PLFfS_dO6IGTlCfz8EHQ3v9LRtrGPuSsLq8Q5CniFPRzgff2_EhT8tLHGLI4hAqVKLNZIF1r1wzwA14GZfIx_YeTQmCWJNxW_UxnotIIkC5IZb59_pZXPP6iCksV8_8J6BQQ3-DD0VET-FpvfVYj7VEAAAACL43U4lbRNrtNGB8RkPol2NWH9swWU1hRKlDZSm8xGjgGlZozGN4vE5x5sMamYAEKPZ1uPVZiCHyVmCb2f7GBx",
"token_type": "bearer",
"expires_in": 86399
}
[/code]
From this point on you have to pass this bearer token with all your queries for successful authentication. In the test application Postman this means that you have to choose the authentication type “Bearer Token”, and copy/paste the above long string (without quotation marks) to the field that appears. This will add one key-value pair to the query header like this:
Authorization Bearer AQAAANCMnd8BFdERjH…
Initializing and using a Neural Network
The parameters POST query requires a Json body that contains the arguments necessary to initialize a neural network – and this includes the training dataset as well. The below example shows a network initialization with 2 hidden layers with 16 neurons, 1 inputs and 5 outputs. Also the network has Linear activation on the input layer (there are no neurons on the input layer), and TanH activation on all the other layers. The selected algorithm is #4, which is Distributed Resilient Propagation in the enum. All these numbers are represented by enums in the code. The training data must be a CSV file dumped into a string.
This is how the body of the message is supposed to look:
[code language=”java”]
{
"neuronCounts": [
1,
16,
16,
5
],
"activationFunctions": [
-3,
1,
1,
1
],
"algorithmType": 4,
"csvData": "-50,2500,-125000,6250000,-312500000,7.071067812rn-49,2401,-117649,5764801,-282475249,7rn-48,2304,-110592,5308416,-254803968,6.92820323rn-47,2209,-103823,4879681,-229345007,6.8556546rn-46,2116,-97336,4477456,-205962976,6.782329983rn-45,2025,-91125,4100625,-184528125,6.708203932rn-44,1936,-85184,3748096,-164916224,6.633249581rn-43,1849,-79507,3418801,-147008443,6.557438524rn-42,1764,-74088,3111696,-130691232,6.480740698rn-41,1681,-68921,2825761,-115856201,6.403124237rn-40,1600,-64000,2560000,-102400000,6.32455532rn-39,1521,-59319,2313441,-90224199,6.244997998rn-38,1444,-54872,2085136,-79235168,6.164414003rn-37,1369,-50653,1874161,-69343957,6.08276253rn-36,1296,-46656,1679616,-60466176,6rn-35,1225,-42875,1500625,-52521875,5.916079783rn-34,1156,-39304,1336336,-45435424,5.830951895rn-33,1089,-35937,1185921,-39135393,5.744562647rn-32,1024,-32768,1048576,-33554432,5.656854249rn-31,961,-29791,923521,-28629151,5.567764363rn-30,900,-27000,810000,-24300000,5.477225575rn-29,841,-24389,707281,-20511149,5.385164807rn-28,784,-21952,614656,-17210368,5.291502622rn-27,729,-19683,531441,-14348907,5.196152423rn-26,676,-17576,456976,-11881376,5.099019514rn-25,625,-15625,390625,-9765625,5rn-24,576,-13824,331776,-7962624,4.898979486rn-23,529,-12167,279841,-6436343,4.795831523rn-22,484,-10648,234256,-5153632,4.69041576rn-21,441,-9261,194481,-4084101,4.582575695rn-20,400,-8000,160000,-3200000,4.472135955rn-19,361,-6859,130321,-2476099,4.358898944rn-18,324,-5832,104976,-1889568,4.242640687rn-17,289,-4913,83521,-1419857,4.123105626rn-16,256,-4096,65536,-1048576,4rn-15,225,-3375,50625,-759375,3.872983346rn-14,196,-2744,38416,-537824,3.741657387rn-13,169,-2197,28561,-371293,3.605551275rn-12,144,-1728,20736,-248832,3.464101615rn-11,121,-1331,14641,-161051,3.31662479rn-10,100,-1000,10000,-100000,3.16227766rn-9,81,-729,6561,-59049,3rn-8,64,-512,4096,-32768,2.828427125rn-7,49,-343,2401,-16807,2.645751311rn-6,36,-216,1296,-7776,2.449489743rn-5,25,-125,625,-3125,2.236067977rn-4,16,-64,256,-1024,2rn-3,9,-27,81,-243,1.732050808rn-2,4,-8,16,-32,1.414213562rn-1,1,-1,1,-1,1rn0,0,0,0,0,0rn1,1,1,1,1,1rn2,4,8,16,32,1.414213562rn3,9,27,81,243,1.732050808rn4,16,64,256,1024,2rn5,25,125,625,3125,2.236067977rn6,36,216,1296,7776,2.449489743rn7,49,343,2401,16807,2.645751311rn8,64,512,4096,32768,2.828427125rn9,81,729,6561,59049,3rn10,100,1000,10000,100000,3.16227766rn11,121,1331,14641,161051,3.31662479rn12,144,1728,20736,248832,3.464101615rn13,169,2197,28561,371293,3.605551275rn14,196,2744,38416,537824,3.741657387rn15,225,3375,50625,759375,3.872983346rn16,256,4096,65536,1048576,4rn17,289,4913,83521,1419857,4.123105626rn18,324,5832,104976,1889568,4.242640687rn19,361,6859,130321,2476099,4.358898944rn20,400,8000,160000,3200000,4.472135955rn21,441,9261,194481,4084101,4.582575695rn22,484,10648,234256,5153632,4.69041576rn23,529,12167,279841,6436343,4.795831523rn24,576,13824,331776,7962624,4.898979486rn25,625,15625,390625,9765625,5rn26,676,17576,456976,11881376,5.099019514rn27,729,19683,531441,14348907,5.196152423rn28,784,21952,614656,17210368,5.291502622rn29,841,24389,707281,20511149,5.385164807rn30,900,27000,810000,24300000,5.477225575rn31,961,29791,923521,28629151,5.567764363rn32,1024,32768,1048576,33554432,5.656854249rn33,1089,35937,1185921,39135393,5.744562647rn34,1156,39304,1336336,45435424,5.830951895rn35,1225,42875,1500625,52521875,5.916079783rn36,1296,46656,1679616,60466176,6rn37,1369,50653,1874161,69343957,6.08276253rn38,1444,54872,2085136,79235168,6.164414003rn39,1521,59319,2313441,90224199,6.244997998rn40,1600,64000,2560000,102400000,6.32455532rn41,1681,68921,2825761,115856201,6.403124237rn42,1764,74088,3111696,130691232,6.480740698rn43,1849,79507,3418801,147008443,6.557438524rn44,1936,85184,3748096,164916224,6.633249581rn45,2025,91125,4100625,184528125,6.708203932rn46,2116,97336,4477456,205962976,6.782329983rn47,2209,103823,4879681,229345007,6.8556546rn48,2304,110592,5308416,254803968,6.92820323rn49,2401,117649,5764801,282475249,7rn50,2500,125000,6250000,312500000,7.071067812rn"
}
[/code]
The GET query parameters returns the exact same model/body in Json format.
When the neural network is initialized properly, the iterations are supposed to be triggered manually with the Learn query, like this:
http://localhost:9000/api/neuralnetwork/learn/10
The above call will execute 10 iterations, and you can continue training after this call by making more learn requests as necessary. The result will be returned as a list of Mean Squared Error values after each iteration, in this case 10 numbers:
[code language=”java”]
[
0.12983532120833302,
0.084435276714808286,
0.048709316197598336,
0.092573188485042882,
0.064019571805354392,
0.08048526485367577,
0.06801106911705862,
0.061548624040171654,
0.067560794071690869,
0.061314801381218552
]
[/code]
The state GET query will return you a state model that describes the current state of the neural network, you can save this model and load it back at a later point with the state POST query:
[code language=”java”]
{
"inputCount": 1,
"outputCount": 5,
"currentIteration": 0,
"algorithmType": 4,
"normalizationLowerBound": -0.9,
"normalizationUpperBound": 0.9,
"learnRate": 0,
"momentum": 0,
"l2Lambda": 0,
"updateIncrement": 0,
"updateDecrement": 0,
"updateMinimum": 0,
"updateMaximum": 0,
"dataFilePath": null,
"layers": [
{
"activationFunction": 1,
"numberOfNeurons": 16,
"weightMatrix": [ 0.298106432, 0.0323681235, 0.07529336, 0.7443464, -0.221400321, 0.150222421, 0.484664559, -0.03762138, 0.0276177526, 0.469199419, -0.772850037, 0.295443177, -0.8536253, 0.367495179, -0.888695657, -0.203839 ],
"biasVector": [ 0.0102121886, 0.134540275, 0.888780534, 0.966368139, 0.7871117, 0.6308095, 0.782798767, 0.5778677, 0.351391017, 0.6637396, 0.0413430668, 0.3402118, 0.2273598, 0.428240269, 0.779353, 0.296584457 ]
},
{
"activationFunction": 1,
"numberOfNeurons": 16,
"weightMatrix": [ 0.8373345, 0.492336273, -0.82880795, -0.6027727, 0.6708245, -0.305496335, 0.62352705, -0.06177813, 0.0755190849, -0.06557912, 0.7956393, 0.6485505, 0.64356637, 0.6942172, 0.40798223, 0.444085121, -0.410786569, -0.497542232, 0.3239267, 0.605489, 0.218772173, -0.5052074, 0.899459958, 0.558400154, 0.0430481434, 0.213340163, 0.630601645, -0.437545, -0.04797715, -0.320077181, 0.6324258, -0.3194272, -0.7016152, 0.470245242, -0.73733443, -0.259154379, -0.5414363, 0.19327569, 0.301633954, -0.664316952, -0.795979, -0.7022486, -0.0584392548, -0.389430523, 0.828135, -0.323353827, 0.0532287359, -0.485260755, -0.42186, 0.78214407, 0.471125, -0.8517409, 0.0189334154, 0.877734661, -0.8071206, -0.182179153, -0.8644647, -0.447037429, -0.7555779, -0.56212306, -0.114421785, -0.26776576, -0.0227239132, 0.6170577, 0.188573718, -0.438037276, -0.5816598, 0.6650027, -0.112794638, -0.6875407, -0.406089067, 0.589189053, 0.228753328, 0.8772522, 0.497182131, 0.346755743, -0.857803166, -0.5036077, -0.7920956, 0.8897301, -0.8855945, 0.845569849, -0.453070939, 0.0267128944, 0.336757421, -0.834835649, -0.310953557, -0.240489, -0.566914558, 0.817225456, -0.06162989, -0.8481303, 0.7429682, -0.4289284, 0.487417817, 0.0301729441, -0.005685091, 0.392222524, 0.0255035162, 0.211954117, 0.8940108, -0.764736, -0.824597538, -0.724354148, -0.465287656, -0.182580054, 0.6334059, -0.205229938, -0.7390329, -0.7133583, -0.6548346, -0.128356159, -0.108718276, 0.515113235, 0.7751496, 0.261051655, -0.262580276, 0.5365168, 0.581521869, 0.36104846, 0.171406507, -0.155057788, -0.185354352, 0.0756471157, 0.3660413, 0.641894937, -0.119748175, -0.6785541, -0.820750535, -0.47947678, -0.163719535, 0.4209361, -0.5270027, -0.4849116, 0.8588631, 0.343779564, -0.5813333, 0.0340098739, -0.249837279, -0.663809836, -0.6293788, 0.000128626823, 0.465062618, -0.615651548, 0.785773039, 0.519481063, 0.238194346, -0.28304714, -0.009735465, -0.09030324, 0.274662733, -0.6401148, 0.70581305, 0.7783636, 0.6748923, 0.8119458, 0.5591291, 0.569976, 0.266026974, 0.3737054, 0.04953158, -0.512282848, 0.777135968, 0.721053243, 0.2569797, -0.09846014, 0.8488709, 0.7216871, 0.224019408, 0.263146639, -0.1862092, -0.768479347, -0.547329068, -0.252190769, 0.5763233, -0.069765985, -0.557971358, -0.472192764, -0.6697641, -0.194338918, 0.422025681, 0.7409276, 0.422523141, -0.4127596, 0.03544295, 0.234447122, -0.8872041, 0.425998926, -0.603532434, -0.791503847, 0.1242404, 0.8299959, 0.326924324, -0.452123672, -0.49185437, 0.482470632, -0.7384476, -0.6729726, -0.491824329, -0.320886135, 0.471314669, 0.4205377, 0.8269801, -0.8793594, 0.1587404, 0.554686546, -0.340990543, 0.406726956, 0.232043982, -0.2089454, 0.7243295, -0.1907584, -0.8166666, -0.446639121, 0.201789618, 0.147254467, 0.5246812, 0.5040214, -0.445569128, -0.776273131, 0.8108913, 0.338054061, 0.738543153, -0.478453219, -0.160648763, 0.8265896, -0.0275793076, 0.265619755, -0.845295668, 0.09092772, 0.820947766, -0.5253651, 0.561421037, -0.385319054, 0.696925163, -0.8849004, 0.1118418, -0.6852272, 0.08569574, -0.6148039, 0.391706467, -0.359362, 0.128069878, 0.445682883, -0.257329762, 0.278509974, -0.156292737, -0.7615964, -0.496775776, 0.854322553, -0.2904846, -0.4753825, 0.147805214, 0.70470953, 0.0407395363, 0.638805747 ],
"biasVector": [ 0.571559, 0.9882679, 0.169947281, 0.196818069, 0.271266818, 0.0748822242, 0.144500479, 0.4785206, 0.06899491, 0.776142061, 0.6397362, 0.7357453, 0.401104271, 0.2576986, 0.386587054, 0.6834658 ]
},
{
"activationFunction": 1,
"numberOfNeurons": 5,
"weightMatrix": [ 0.481178045, 0.508251667, 0.7180277, 0.319514751, -0.368593633, 0.514815569, 0.0794328451, -0.175002635, 0.4648763, 0.3864665, 0.201710582, 0.8309537, -0.764376044, -0.042178452, -0.378389, -0.2880209, 0.640696764, -0.5247639, -0.732267439, -0.6855736, 0.347237468, 0.112083077, 0.431229234, 0.330511332, 0.4352491, 0.001793325, 0.487529755, 0.5435344, -0.7449354, 0.6386188, -0.275335968, -0.332265675, 0.163447976, 0.2836603, 0.485178828, -0.1293599, 0.164735079, 0.386372924, 0.085924685, -0.067949, -0.04347557, -0.4952125, -0.05979091, -0.77840513, 0.380881667, 0.389255524, -0.852278948, -0.345734239, -0.15316838, -0.0276187062, 0.08341819, 0.7077552, 0.435510278, -0.144388556, -0.6433679, 0.5018159, -0.3275953, -0.3985957, 0.795333862, 0.6885598, -0.6508402, 0.3261119, 0.5049691, -0.0013474822, 0.232979655, -0.08225691, -0.368610084, -0.36327827, 0.7599044, -0.545953155, 0.2237513, 0.279692173, 0.5388886, -0.198280752, -0.7983859, -0.450310439, 0.5097828, 0.599840164, -0.6194632, 0.5113611 ],
"biasVector": [ 0.942466736, 0.124681249, 0.481427431, 0.6583923, 0.21833165 ]
}
]
}
[/code]