Factoring the "0A" key

[Skittle]

I have just started 22m-25m on a Q6600 @ 3.0ghz
:)

EDIT: just replaced with 64bit binaries !

~little over 10% done
edit2: forgot i had computer set to sleep after 2 hours, lost about 6 hours of crunching. now about 20%
edit3: about 37% | 1486322 relations
43% done | 1905023 relations
70% done | 2873950 relations
80% done | 3292436 relations
looks like it will be done in the early morning ~ 7 hours

[squalyl]

FloppusMaximus:

The BOINC server is up and running, I used a test application to validate this part.

The BOINC application is ALMOST ok, the code is adapted to use Boinc, everything works (workunits generation, download to client, siever execution) EXCEPT I get a strange behaviour inside boinc because the computation is long, and I don't "talk" with the BOINC manager during this time, so it thinks the application is dead and refuses to upload the result although the work is done! (wow fck*** long sentence, sorry X_X)

We get this error: http://www.boinc-wiki.info/Result_%27%28re...inished%27_file

I've spotted the point in the code where the
[font="Courier New"]fprintf(stderr, "\rtotal yield: %u, q=%u (%1.5lf sec/rel)", (unsigned int)yield, (unsigned int)special_q, (tNow - tStart)/yield);

[Skittle]

[FloppusMaximus]

Skittle: No, as it says, that only affects polynomial selection. We've already selected the polynomial (it's in the k0A.txt file that I posted); now we need to find relations.

(If you're curious, factMsieve does not use multiple cores for the polynomial selection phase. If you wanted to, you could hack around and use the pol5 tools directly, or you could tweak the script and perform several searches simultaneously, starting at different coefficient values.)

And thanks for your help!

Squalyl: Sounds exciting. I'm looking forward to it.

[squalyl]

That's definitely exciting.

We could also use BOINC for the polynomial selection, the only programming to do would be to wrap the right programs and a selector to keep the best reported result.

I'm progressing on the siever part. I think the problem existed because I did not report any progress during the computation.

I don't know if you looked at the interesting code, but I believe I can get a fraction of the total job by calculating:

[font="Courier New"]boincstatus(((double)(special_q - first_spq))/((double)sieve_count));

[squalyl]

Okay everyone the boinc application is fixed, I didn't notice some exit() calls in the middle of computation function, which prevented execution of the boinc_finish() call. The BOINC client took this for an application crash.

Here are the informations to participate to the effort if you're running windows.

First, download BOINC.
http://boinc.berkeley.edu/download.php

Then, attach to this project
http://boinc.unsads.com/rsals/

You can register using the boinc client.

And that's all.
note that there's no cruncher for linux yet.

I've added the key_0A to the list, tell me what ranges you want to work on and I'll add some workunits.

The polynomial I have is this one:

[Lionel Debroux]

Well, I can only echo squalyl's congratulations to Benjamin and the others... and we can congratulate him, too

[squalyl]

Hehe:

[font="Courier New"]$ ./my_gnfs_make_wus.php input/key_ti83p.job k0A 25000000 28000000 10000
generating work for k0A from 25000000 to 28000000, count = 10000
...
done, created 300 work units

[magicdanw]

magicdanw's Linux computer is sad he cannot help

[Lionel Debroux]

Linux binaries should be available within a day or two

[squalyl]

I'm working on it right now :)

edit: done, the binary is statically linked and should work on any linux with a cpu >= pentium 3.

I really need reports for this application, I just recompiled the windows code for linux, hope it works OK.

Just enable your boinc client.

edit: just added 200 WUs for the k0A key in the 28M-30M range.

[Skittle]

BOINC client works on debian, however my laptop does not have sufficient ram to run any it.

[magicdanw]

Nice work, Linux seems to be handling the jobs well!

[FloppusMaximus]

squalyl: Sounds great! And thank you very much for hosting this and getting it working.

Have you dealt with the issue of re-calculating the algebraic factor base? For q-values below 27 million (i.e., the AFB limit specified in the poly file) you need to lower alim appropriately and regenerate the AFB. For q-values above 27 million, you can reuse the AFB and save a bit of time. Have a look at what factMsieve does - I think it just deletes the AFB cache file after running the siever, whenever Q0 < alim.

I don't want to be a spoilsport here, but please don't spend a lot of CPU time on this before we're sure it is working.

Is the source for your BOINC client available somewhere? What would I need to do to compile it?

Also, I'm done with 20M-22M. I guess I'll take 25M-28M for now.

[squalyl]

To get the cruncher working I modified some ggnfs files.
there's no perl here, just a modified form of gnfs-lasieve4I14e
What I do is resolve file names, construct the command line, and call the original code.
Every search job is independent from others, no state is retained inbetween.

My repository is here:

https://www.unsads.com/scm/svn/squalyl/boinc/rsals

login guest , password guest

The modified files include a new project under build.vc, and also src/lasieve4/Makefile and src/lasieve4/ggnfs-lasieve4e.c

a new binary target was added to the makefile to link with boinc libraries (link phase needs g++ since boinc code has some c++)

to build, you need gmp (the one provided is for win32, there should be a gmp-devel like package in every distro) and boinc, here is how I got it:
[font="Courier New"]
svn co -r 18818 http://boinc.berkeley.edu/svn/trunk/boinc boinc
cd boinc
./configure --prefix=/usr --disable-client --disable-server
make
sudo make install

[Skittle]

Since qintsize is set to (default) 100000 and factmsieveQ can only launch one instance of gnfs-lasieve4I14e.exe at a time, i have cores sitting idle because they are waiting for the last gnfs-lasieve4I14e.exe to finish before moving on. would setting qintsize to a smaller value (20000) help? and if so can i safely stop the process, change the parameters and continue?

[FloppusMaximus]

Skittle: I doubt that would help much, but you can certainly try it if you want to. What would probably work better, if you want to try it, would be to run multiple instances of the script simultaneously, on different q ranges (you might have to run them in different directories.)

squalyl: Thanks. I will have a look at your code later tonight; maybe I'll try to adapt the experimental siever, so we can have a fast 64-bit client. Would BOINC allow for multiple sub-architectures (e.g. separate pentium3/pentium4 builds)? I don't know how much difference that would make.

I'm not sure about the AFB issue - the BOINC version doesn't seem to be saving the AFB at all, but perhaps I'm missing something. It would certainly save some time, especially with a small qintsize, if we could reuse the AFB when sieving large q values.

[FloppusMaximus]

Oops. Squalyl, I saw your message about working on 28M-30M and missed the part about 25M-28M. I will work on 30M-33M instead.

[fullmetalcoder]

17M-20M : done -> 4161960 relations

I'm on 10M-13M now.

[FloppusMaximus]

fullmetalcoder: Great, thanks!

squalyl: Here's what I've found so far.
- The siever was not saving the factor base because main() calls boincstart() with argc_init = 0, so the first "argument" (which happens to be -k) is written as argv[0], hence ignored.
- At the same time, we should not be using the -k option until we work out how to deal with the FB files. In particular, if you use the -k option as is, FB files will just get dumped in the project directory, and, as I understand it, these files will never get cleaned up.
- The -R option only works if you use -f and -c to specify the q range. It doesn't work if you get the q range from a job file. From what I can figure, though, it should be safe to move the "resume" code down to below the call to parseJobFile(). (The code is so messy that whoever implemented the resume function probably didn't notice what was going on there.)
- I think I made all the necessary changes to the experimental siever. It seems to work in "standalone" mode, anyway.

Patch: [attachment=2739:rsals_fixes.patch.txt]
(For reference, the experimental siever seems to run about twice as fast as the athlon-xp siever, which is only a little bit faster than the pentium3 siever.)

As far as the AFB files go, here are a few options:
1. Don't save the AFB to a file at all, and just don't worry about it. It's not a huge amount of time we're wasting at the moment.
2. For work units with q0 ≥ alim, have the siever reuse an existing AFB file, or generate a new one if it doesn't exist. This would require modifying the siever to use a BOINC-specified AFB file name; that's not hard. On the BOINC side, I think it would be sufficient to add a second input file reference with the 'generated_locally' and 'sticky' attributes. But this would have to be done only when q0 ≥ alim.
3. Have somebody generate and upload all of the complete AFB files. Then, for work units with q0 ≥ alim, have clients download the corresponding AFB file as well as the job file. These files will be around 13 megabytes, but the client should only have to download each of them once. Again, this would require a slight modification to the siever. This would not work for clients on big-endian systems.