neural network - How to correctly accumulate gradients across batches in Torch? -


i'd accumulate gradients across several batches. training iter_size 2 , batch_size 16 should same if set iter_size = 1 , batch_size = 32. suspect there i've missed in code, because gradparams both cases not same. appreciate if me find out problem. here code:

   local params, gradparams = net:getparameters()    local iter_size = 2    local batch_size = 16    local iter = 0    net:zerogradparameters()    i, input, target in trainset:sampleiter(batch_size)       iter = iter + 1       -- forward       local input = input:cuda()       local target = target:cuda()       local output = net:forward(input)       local loss = criterion:forward(output, target)       local gradoutput = criterion:backward(output, target)       local gradinput = net:backward(input, gradoutput)       -- update       if iter == iter_size           gradparams:mul(1.0/iter_size)           net:updategradparameters(0.9)           net:updateparameters(0.01)           iter = 0           net:zerogradparameters()       end    end

it worth mentioning manually set random seed determinism when comparing results, difference not due random initialization of network.

the problem due sampling, sampleiter returned images in different order different batch sizes, batches in these 2 cases contained different images , accumulated gradients different.


-

Popular posts from this blog

Apache NiFi ExecuteScript: Groovy script to replace Json values via a mapping file -

i working apache nifi 0.5.1 on groovy script replace incoming json values ones contained in mapping file. mapping file looks (it simple .txt): header1;header2;header3 a;some text;a2 i have started following: import groovy.json.jsonbuilder import groovy.json.jsonslurper import java.nio.charset.standardcharsets def flowfile = session.get(); if (flowfile == null) { return; } flowfile = session.write(flowfile, { inputstream, outputstream -> def content = """ { "field1": "a" "field2": "a", "field3": "a" }""" def slurped = new jsonslurper().parsetext(content) def builder = new jsonbuilder(slurped) builder.content.field1 = "a" builder.content.field2 = "some text" builder.content.field3 = "a2" outputstream.write(builder.toprettystring().ge...
Read more

python 3.x - PyQt5 - Signal : pyqtSignal no method connect -

following example in doc found on sourceforge faced error. from pyqt5.qtcore import pyqtsignal, pyqtslot def setsignal(): signal = pyqtsignal() signal.connect(self.myaction) @qtcore.pyqtslot() def myaction(): print("signal triggered") results in attributeerror: 'pyqt5.qtcore.pyqtsignal' object has no attribute 'connect' as mentioned in docs linked, signal needs defined on class level: class foo(qobject): signal = pyqtsignal() def connectsignal(): self.signal.connect(self.myaction) @qtcore.pyqtslot() def myaction(): print("signal triggered")
Read more