Factoring the "0A" key

[ShadowPhoenix]

[Skittle]

[limx]

Catting them together should work, if the order of the filenames is right you can try:

[FloppusMaximus]

Yep, you can just cat the files together. The order doesn't matter. ShadowPhoenix is currently working on it with his monster Mac, so we may not be needing your CPU after all.

[FloppusMaximus]

Don't drop those pencils yet! Word from relation filtering is that we still need more relations. Another couple million should do it.

Squalyl, are we still waiting for any more results from BOINC? If not, could you put in another chunk of workunits, please?

[ShadowPhoenix]

It seems that those relations require monstrous amounts of RAM.(6gigs?)

As for relations it seems (if I am MAYBE reading this correctly) that we need 1.2M more "clean" realtions. Right now we have 17M?

Assuming the majority of work was done in the past 5 days that signifies abotu ~10% more... so 12 hours.

[dga]

If you need more (or even think you might need more) relations, you might add them sooner rather than later. I've added 76 cores from my cluster while we're in the process of moving to a new cluster management system, but we'll probably be taking it down in a week or two for the switchover, after which the machines will no longer be idle. In the meantime, please feel free to abuse the heck out of them. It looks like you're set as far as machines for the final computation, but drop me a note if you end up short. Plenty of 8 core / 16GB dram machines to lend for a few days if needed. I'm happy to support your project for a while, though human-time to do so is going to be very limited.

[FloppusMaximus]

dga: Wow, thanks! Right now the easiest way you can help is just running the BOINC client. But if you do want to work on final processing, there are plenty more keys to go around.

It looks like results are still coming in for the 0A key. When ShadowPhoenix tried running Msieve a few hours ago, it claimed that it needed 1166409 more relations. My experience has been that these estimates are never very accurate. But we are very close at this point - it got to the last stage of relation filtering with 760062 surplus relations (out of 990401 needed.)

[ShadowPhoenix]

@dga:

Thank you so much for your work! I can do everything needed for final computation on yours (if you don't mind) if you can provide me with a user/IP. All that's needed is download some data from HTTP, uncompress, cat it together, download msieve, make msieve (no admin rights needed, it gcc's it in place), and then run msieve.

I don't think you even need to put the user into any groups. No su/sudo, only internet access and gcc access. And about 20 gigs of space. (maybe 10 if you can't do 20)

Or I can make a python script to do all that so you can inspect source code . Whichever way you prefer.



If anyone is willing to put up new BOINC result I am ready to try it out now or a bit later when dga reports back.

EDIT: omg, it's 2AM! New data will be downloaded in 10 minutes. 28 minutes and I will let everyone know.

[FloppusMaximus]

There should be a new version of 'all_ti84p.tar' up by now. If you want to be a little kinder to the server, note that in the other thread, Lionel posted some commands you can use to download only those result files you don't already have.

[ShadowPhoenix]

[FloppusMaximus]

Here: http://www.unitedti.org/index.php?showtopi...st&p=135741

[ShadowPhoenix]

[FloppusMaximus]

It sounds like the .fb and .ini files got mixed up somehow. The .ini file should just contain one line (the number to factor.) The .fb file should contain a bunch of polynomial coefficients and sieving parameters.

Your command line looks correct...

[ShadowPhoenix]

MD5 (k0A.ini) = 140ff3efa5f3282b1e488cf9a9a782f1
MD5 (k0A.fb) = 140ff3efa5f3282b1e488cf9a9a782f1

cat k0A.ini
[indent]N 12537072476862421230323818220670094176024121433129402568805703301509542532518840
117998301702770251376727694567865443226427097771931350481518184606521695409
SKEW 514644.23
A5 7797600
A4 23599061408952
A3 87576392702571674174
A2 -346231005879880708232825
A1 -3621573065566539657182186001324
A0 -174992518730891284837277302704395277
R1 640915374563362171
R0 -276214470369449464279330486112
FAMAX 27000000
FRMAX 27000000
SALPMAX 536870912
SRLPMAX 536870912[/indent]


Edit: re-doing it with normal .ini file. Typed it in by hand

So, be back in 15 mins and I will tell if it works now.

[FloppusMaximus]

Oops :)

head -1 k0A.fb | awk '{print $2}' > k0A.ini

[ShadowPhoenix]

Ah much better. Now it doesn't try to factor the Rvalues. Instead I get...

[indent]
commencing in-memory singleton removal
begin with 18302760 relations and 17234762 unique ideals
reduce to 18044029 relations and 16975451 ideals in 15 passes
max relations containing the same ideal: 25
filtering wants 1000000 more relations
elapsed time 00:19:19

[/indent]


Weird! We only gained 166 409 relations in one full day of BOINC? Or did we update BOINC prematurely yesterday?

Either way, don't give up, we'll get them soon! Focus on 84p! All hands on deck!

[FloppusMaximus]

How are you counting? You didn't say how many there were before filtering, but after filtering, we've gained 909639 relations and 601123 ideals - i.e., a net gain of 308516.

Edit: Oh, sorry, you're looking at the last line there. Yeah, when Msieve says "filtering wants 1000000 more" it just means it can't figure out how many more it wants. :)

Apparently it wants more excess than it needed for the 04 key. But we also have a lot more small ideals, since we've been sieving the small q values.

[ShadowPhoenix]

[FloppusMaximus]

Small ideals aren't better or worse than large ones (well, OK, they're better if they make it easier to find relations.) My point was just that Msieve is expecting a different distribution of ideals than what we have. I don't think this is a major problem, but it is a rather complicated business, and I can't claim to understand it all.

The main goal is simply for us to have more relations than ideals (once that happens, we can build a matrix, solve for linear dependencies, and hopefully, derive a factorization from one of those dependencies.) A secondary goal is for the matrix to be as small as possible (so that it's easier to solve.) The singleton removal phase serves both of these purposes, by removing unhelpful relations. You'll notice that Msieve performs singleton removal in two steps; in the first step, it only looks at "large" ideals above some limit (e.g., 40 million); in the second step, it adds smaller (but still rather large) ideals into the mix. But it always ignores ideals smaller than 7.2 million, and it also ignores ideals that occur very frequently; these ideals figure into the "target excess". So as we add more relations, the target excess will continue to increase (though, hopefully, the actual excess will increase faster.) At any rate, that's the best I can gather from reading bits of the Msieve source.