20210919, 05:48  #254  
"Jane Sullivan"
Jan 2011
Beckenham, UK
115_{16} Posts 
I've just run version 2.07 on my Win7 machine, and I'm pleased to say that the trouble I was having with 2.06 does not happen with 2.07.
Code:
JANELT3 C:\Users\Jane\Documents\Maths\yafu\Versions\yafu2.07 > yafux64 YAFU Version 2.07 Built with Microsoft Visual Studio 1928 Using GMPECM 7.0.4, Powered by MPIR 3.0.0 Detected Intel(R) Core(TM) i74710MQ CPU @ 2.50GHz Detected L1 = 32768 bytes, L2 = 6291456 bytes, CL = 64 bytes Using 1 random witness for RabinMiller PRP checks Cached 664579 primes; max prime is 9999991 =============================================================== ======= Welcome to YAFU (Yet Another Factoring Utility) ======= ======= bbuhrow@gmail.com ======= ======= Type help at any time, or quit to quit ======= =============================================================== >> nfs(296461670302265261661404037493475256820003211696331293471947264729761541) starting SIQS on c72: 296461670302265261661404037493475256820003211696331293471947264729761541 ==== sieving in progress ( 4 threads): 17568 relations needed ==== ==== Press ctrlc to abort and save state ==== 17744 rels found: 6988 full + 10756 from 123762 partial, (2492.55 rels/sec) SIQS elapsed time = 53.9072 seconds. ***factors found*** P31 = 5080439387107487549436920600141 P41 = 58353549312012091454346752496532032405401 ans = 1 >> exit Quote:


20210919, 12:42  #255  
"James Heinrich"
May 2004
exNorthern Ontario
3,499 Posts 
NFS is even more opaque (to those like me who don't really understand what it's doing). It outputs a lot of stuff to the screen, such as
Quote:
Last fiddled with by James Heinrich on 20210919 at 12:49 

20210920, 15:11  #256 
"Ed Hall"
Dec 2009
Adirondack Mtns
3998_{10} Posts 
Well, I spoke too quickly about all being well. (That should teach me a lesson, but probably won't.)
First, all my AMD machines are failing when they they try to use snfs to solve. I've temporarily sidestepped that trouble by using "snfs_xover=95." Second, I'm still having the looping issues with SIQS. At present, I have been logging a single episode with v v v and have a log file that's over 25MB. (It's been looping for nearly 11 hours.) I've also discovered that YAFU is currently using 22GB of RAM, and growing into swap. I've tried to trim the log down to the repeating section to make it fit a code block. Here is a portion of the "top" display: Code:
Tasks: 433 total, 3 running, 430 sleeping, 0 stopped, 0 zombie %Cpu(s): 0.5 us, 0.3 sy, 0.2 ni, 92.8 id, 6.1 wa, 0.0 hi, 0.0 si, 0.0 st MiB Mem : 24041.0 total, 324.9 free, 23343.9 used, 372.2 buff/cache MiB Swap: 16373.0 total, 9720.1 free, 6652.9 used. 308.1 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 2911573 math98 20 0 30.3g 22.0g 1212 R 7.0 93.5 627:21.54 yafu Code:
09/20/21 00:18:25, ===================================== 09/20/21 00:18:25, System/Build Info: 09/20/21 00:18:25, YAFU Version 2.07 09/20/21 00:18:25, Built with GCC 9 09/20/21 00:18:25, Using GMPECM 7.0.5dev, Powered by GMP 6.2.1 09/20/21 00:18:25, detected Intel(R) Xeon(R) CPU X5650 @ 2.67GHz detected L1 = 32768 bytes, L2 = 12582912 bytes, CL = 64 bytes 09/20/21 00:18:25, using 1 random witness for RabinMiller PRP checks 09/20/21 00:18:25, Cached 664579 primes: max prime is 9999991 09/20/21 00:18:25, Random seed: 13035443370557393521 Code:
found 72442 cycles from 160635 relations in 5 passes distribution of cycle lengths: length 1 : 23003 length 2 : 16947 length 3 : 13162 length 4 : 8489 length 5 : 5159 length 6 : 2853 length 7 : 1460 length 9+: 1369 largest cycle: 17 relations matrix is 68528 x 72442 (16.8 MB) with weight 3819360 (52.72/col) sparse part has weight 3819360 (52.72/col) filtering completed in 4 passes matrix is 59776 x 59840 (13.3 MB) with weight 3016591 (50.41/col) sparse part has weight 3016591 (50.41/col) saving the first 48 matrix rows for later matrix is 59728 x 59840 (11.4 MB) with weight 2594461 (43.36/col) sparse part has weight 2394272 (40.01/col) matrix includes 64 packed rows using block size 23936 for processor cache size 12288 kB commencing Lanczos iteration memory use: 9.4 MB lanczos halted after 945 iterations (dim = 59721) recovered 14 nontrivial dependencies Lanczos elapsed time = 14.0724 seconds. Sqrt elapsed time = 1.0826 seconds. SIQS elapsed time = 15.7047 seconds. pretesting / qs ratio was 1.05 fac: setting target pretesting digits to 27.38 t15: 73.14 t20: 33.35 t25: 4.59 t30: 0.48 t35: 0.04 fac: estimated sum of completed work is t27.39 starting SIQS on c89: 11539488294330117240202826559598428302679556688248012471560145814831982545559910259149337 input from file = 11539488294330117240202826559598428302679556688248012471560145814831982545559910259149337 input to yafu = 11539488294330117240202826559598428302679556688248012471560145814831982545559910259149337 input matches with multiple of 1 static memory usage: initial cycle hashtable: 16777216 bytes initial cycle table: 160000 bytes factor base: 1370560 bytes fb bounds small: 1024 SPV: 38 10bit: 104 11bit: 176 12bit: 312 13bit: 552 32k div 3: 712 14bit: 1000 15bit: 1760 med: 2880 large: 16688 large_x2: 68528 all: 68528 start primes SPV: 251 10bit: 1063 11bit: 2053 12bit: 4231 13bit: 8221 32k div 3: 11087 14bit: 16421 15bit: 32803 med: 57667 large: 392957 large_x2: 1823713 all: 1823713 memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes memory usage during sieving: curr_poly structure: 1048656 bytes relation buffer: 7602176 bytes factor bases: 184320 bytes update data: 890864 bytes sieve: 32768 bytes bucket data: 1573672 bytes restarting siqs from saved data set 72443 relations found: 23003 full + 49440 from 882638 partial threw away 0 relations with large primes too small ==== sieve params ==== n = 91 digits, 300 bits factor base: 68528 primes (max prime = 1823713) single large prime cutoff: 200608430 (110 * pmax) double large prime range from 3325929106369 to 2287420354104121 DLP MFB = 1.85 using SSE2 enabled 32k sieve core sieve interval: 12 blocks of size 32768 polynomial A has ~ 11 factors using multiplier of 137 using Q2(x) polynomials for kN mod 8 = 1 using SPV correction of 22 bits, starting at offset 38 trial factoring cutoff at 94 bits ==== sieving in progress ( 24 threads): 68592 relations needed ==== ==== Press ctrlc to abort and save state ==== 72443 rels found: 23003 full + 49440 from 882638 partial, (1709269.38 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1709166.16 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1709121.00 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1709075.85 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1709050.04 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1709027.47 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1709001.67 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708979.09 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708953.29 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708930.72 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708904.92 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708879.13 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708856.56 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708830.76 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708808.19 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708782.40 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708692.12 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708663.11 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708640.55 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708614.76 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708592.19 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708566.40 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708543.84 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1708521.28 rels/sec) 72443 rels found: 23003 full + 49440 from 882638 partial, (1677482.63 rels/sec) sieving required 0 total polynomials (4294967295 'A' polynomials) trial division touched 0 sieve locations out of 0 dlpecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful total reports = 0, total surviving reports = 0 total blocks sieved = 0, avg surviving reports per block = nan Elapsed time: 0.5399 sec QS elapsed time = 0.5400 seconds. ==== post processing stage (msieve1.38) ==== read 905640 relations begin singleton removal with 905640 relations reduce to 160635 relations in 11 passes attempting to read and process 160635 relations recovered 160635 relations recovered 127168 polynomials attempting to build 72442 cycles Thanks for any help. 
20210920, 15:30  #257 
"Ben"
Feb 2007
2×1,789 Posts 
What is the command line that starts these looping jobs? I see it is a batchfile job but yafu calls pipes and redirects batchfile jobs also, I think. Wanted to see if I can duplicate here and how yafu is called may matter...
As for your difficulties with older machines I can nearly guarantee it is a yafu issue and not msieve. yafu's siqs is nearly 100% assembly or intrinsics code using sse4.1 or better. The Phenom II x4 doesn't have sse4.1. Yafu's sse2only compile option will apparently still compile but I haven't tested that build in... a while. Yay, glad to hear it! Last fiddled with by bsquared on 20210920 at 15:32 
20210920, 15:35  #258 
"Ed Hall"
Dec 2009
Adirondack Mtns
2×1,999 Posts 
Addendum to previous SIQS issue
I've also noticed that siqs.dat has a current size of over 50MB. I removed it and YAFU broke out of the loop. I've attached the end of the log, since it is too large for a code block. I have saved the larger siqs.dat, but it's too large to post. If you would like, I can work with it in some manner, or maybe compress and post it somewhere.
Thanks! 
20210920, 16:06  #259  
"Ed Hall"
Dec 2009
Adirondack Mtns
7636_{8} Posts 
Quote:
The looping is very intermittent as a whole, so I don't think you'll be able to duplicate it easily. (That's why I have all my machines using "v v v" to a log file.) Last night, I factored several thousand composites in the 89 digit range across several machines and only one looped. (Unfortunately, it takes that machine out of the system until I intervene.) The previous night, during about the same time frame, four looped. YAFU is being called via a BASH script line: Code:
echo "factor(${composite})"  ./yafu one >>dbW.log The siqs.dat file is explicitly deleted prior to each run and the dbW.log file is reinitialized for each run as well, so they are purely written by the current iteration of YAFU. Thanks for all. 

20210920, 16:15  #260  
"Ben"
Feb 2007
110111111010_{2} Posts 
Quote:
So ignore everything I said about sse2/4.1, that shouldn't be an issue with snfs. For snfs the yafuside code should be pretty agnostic to cpu... so I'm not sure why it would not work with older machines. Maybe it's an msieve issue after all. I will take a look at the yafu side at least and see if anything occurs to me. 

20210920, 18:30  #261 
"Evan"
Dec 2020
Montreal
71 Posts 
A couple feature requests/thoughts:
Would it be possible to implement aurifeuillian factor calculation? would speed up factorization of certain forms. Would prioritizing composite factors found via rho/ecm be a feature you'd be able to add? 
20210921, 18:24  #262 
"Ed Hall"
Dec 2009
Adirondack Mtns
2·1,999 Posts 
More loop issue data
So far today I've found four instances (includes overnight). But, again, that's in several thousand factorizations, so it's still not common.
I found some interesting (at least to me) bits in factor.log of one of the machines. I've attached the whole log. Of interest, I found these differences between failed loops and the final one that was after I removed siqs.dat. Here are some lines from the failed loop just previous to the successful one: Code:
09/21/21 12:33:35, DLP MFB = 1.85 09/21/21 12:33:35, using SSE41 enabled 32k sieve core 09/21/21 12:33:35, sieve interval: 13 blocks of size 32768 09/21/21 12:33:35, polynomial A has ~ 12 factors . . . 09/21/21 12:33:35, ==== sieving started ( 8 threads) ==== 09/21/21 12:33:35, trial division touched 0 sieve locations out of 0 09/21/21 12:33:35, total reports = 0, total surviving reports = 0 09/21/21 12:33:35, total blocks sieved = 0, avg surviving reports per block = nan 09/21/21 12:33:35, dlpecm: 0 failures, 0 attempts, 0 outside range, 0 prp, 0 useful 09/21/21 12:33:35, 79066 relations found: 21666 full + 57400 from 1145924 partial, using 0 polys (1 A polys) 09/21/21 12:33:35, on average, sieving found inf rels/poly and 2083055.32 rels/sec 09/21/21 12:33:35, trial division touched 0 sieve locations out of 0 09/21/21 12:33:35, ==== post processing stage (msieve1.38) ==== 09/21/21 12:33:35, QS elapsed time = 0.5606 seconds. . . . 09/21/21 12:33:52, lanczos halted after 1138 iterations (dim = 71861) 09/21/21 12:33:52, recovered 15 nontrivial dependencies 09/21/21 12:33:54, Lanczos elapsed time = 17.6298 seconds. 09/21/21 12:33:54, Sqrt elapsed time = 1.2765 seconds. 09/21/21 12:33:54, SIQS elapsed time = 19.4675 seconds. Code:
09/21/21 12:33:54, DLP MFB = 1.85 09/21/21 12:33:54, allocating 8 large prime slices of factor base 09/21/21 12:33:54, buckets hold 2048 elements 09/21/21 12:33:54, large prime hashtables have 1703936 bytes 09/21/21 12:33:54, using SSE41 enabled 32k sieve core 09/21/21 12:33:54, sieve interval: 13 blocks of size 32768 09/21/21 12:33:54, polynomial A has ~ 12 factors . . . 09/21/21 12:33:54, ==== sieving started ( 8 threads) ==== 09/21/21 12:38:46, trial division touched 39333795 sieve locations out of 521638707200 09/21/21 12:38:46, total reports = 39333795, total surviving reports = 13059246 09/21/21 12:38:46, total blocks sieved = 15919150, avg surviving reports per block = 0.82 09/21/21 12:38:46, dlpecm: 25 failures, 1061012 attempts, 7063329 outside range, 4629850 prp, 860162 useful 09/21/21 12:38:46, 78670 relations found: 21569 full + 57101 from 1143648 partial, using 612275 polys (1100 A polys) 09/21/21 12:38:46, on average, sieving found 1.90 rels/poly and 3985.64 rels/sec 09/21/21 12:38:46, trial division touched 39333795 sieve locations out of 521638707200 09/21/21 12:38:46, ==== post processing stage (msieve1.38) ==== 09/21/21 12:38:46, QS elapsed time = 292.3540 seconds. . . . 09/21/21 12:39:03, lanczos halted after 1139 iterations (dim = 71926) 09/21/21 12:39:03, recovered 16 nontrivial dependencies 09/21/21 12:39:04, prp59 = 52849932024287778885817430875314364339557493580386262089853 09/21/21 12:39:04, prp33 = 166873661278274944173735579427313 09/21/21 12:39:04, Lanczos elapsed time = 17.4163 seconds. 09/21/21 12:39:04, Sqrt elapsed time = 0.3407 seconds. 09/21/21 12:39:04, SIQS elapsed time = 310.1780 seconds. Thanks for all. 
20210921, 19:41  #263  
"Ben"
Feb 2007
2·1,789 Posts 
Quote:
When you manually delete the file then a different set of dependencies would result and (with high likelihood) the factorization completes successfully. Assuming this conclusion is correct, the question now is what to do about it. First, detect it, and second, probably do some more sieving. Maybe step zero is to just print some statement to the log file that no factors were found during sqrt. Then your script can see it and start over. 

20210921, 19:58  #264 
"Ed Hall"
Dec 2009
Adirondack Mtns
111110011110_{2} Posts 
Could I delete siqs.dat as soon as "SIQS elapsed time" appears in factor.log? Is there any need for siqs.dat to exist after that point? It would be easy for me to have a separate script watching factor.log for that line and invoke a deletion. As long as that wouldn't interfere with normal runs, I'll try it. I'm already deleting siqs.dat prior to each run anyway. This would just do it sooner.

Thread Tools  
Similar Threads  
Thread  Thread Starter  Forum  Replies  Last Post 
yafu ignoring yafu.ini  chris2be8  YAFU  6  20191017 16:22 
Running YAFU via Aliqueit doesn't find yafu.ini  EdH  YAFU  8  20180314 17:22 
YAFU1.34  bsquared  YAFU  119  20151105 16:24 
Yafu bug.  storflyt32  YAFU  2  20150629 05:19 
yafu 1.32  bsquared  YAFU  28  20120720 16:17 