what to do with the data?
Author 
Message 
clm
Posted on: Sat Mar 03, 2012 4:04 pm
Posts: 690 Joined: Fri May 13, 2011 6:51 pm

Re: what to do with the data?
giulio wrote: pnm wrote: clm wrote: I will comment first on the “solving rate”. I am a little confused. When you say (page 2):
I looked at the code again, and made a mistake: the division by the size (dimension, e.g. 6 for 6x6) doesn't happen. Patrick So Patrick, can you confirm that you calculate the difficulty as I explained in my recent post (minus the division by the size)? clm, No need to apologise for your interesting post. I welcome your skepticism. Hi, giulio Yes, in Europe we have suffered for centuries the “dogmas” (the truth “revealed”) (“guilty of thinking” … , Giordano Bruno, Miguel Servet, John Huss, … ) until Galileo, Descartes, Darwin, Einstein, … (a good advice for our sons “before going to any Vademecum get some conclusions by yourself”) something happens with the mankind when even today in many parts of the world people prefer the absolutism in its various forms (by the way I have read some of your contributions on these sociological subjects). Well, “let’s retake the leitmotiv of the debate ... “, the “solver rating”. An initial comment: Perhaps all this discussion would have been better in the thread “describe your solver” (in the section “Solving strategies and tips”) rather than in this thread “what to do with the data?” (mainly oriented to the Patrick’s paper, to avoid the deviation from this subject). Initially, we have two different problems: Rating A. Once we have a puzzle with a unique solution: to evaluate the “rating” Rating B. To create a puzzle of a prestablished “rating” (this is more difficult in my opinion). According to the last Patrick’s clarification the division by the size of the puzzle is not obtained, then only averaging the natural logarithm of the possible permutations of every row and column (after applying the restrictions imposed by each cage). So, for a 9x9: SR = [ln(P1) + ln(P2) + … + ln(P18)] / 18 if we name P1 … P9 the number of permutations for the rows 1 thru 9 and P10 … P18 for the columns 1 thru 9. This expression is equivalent to: [ln (P1 x P2 x P3 … x P18)] / 18 = natural log of the root of order 18 of the product of all those permutations P1 … P18. Well, that’s a criterion. In the example you expose, you have 10.1384 as the log for the first row, assuming all 18 rows and columns have a number of permutations in the same range, you would have obtained: SolverRating = 10.2 but not the actual 94.8 assigned to the yesterday’s 9x9 (126 for the february 28’s 9x9, 82.5 for the march 01’s 8x8 and 179.4 for the march 01’s 12x12). You may try different things:  Multiply the final result by 10 (like when calculating decibels … but without a reference) so avoiding a result of zero as in the Patrick’s figure 1 since some combinations at least exist.  Instead of calculating the possible permutations by rows or columns to do that by “cages” (of two or more cells of course) and averaging by the number of those cages (this must be studied more carefully).  You might do some day by day analysis with the Patrick’s 5x5’s to correlate the solving rates and adjust the formula just in case you are looking for something similar. Probably the actual ratings in the Patrick’s page are of the type Rating A. Anyway, till the moment you are doing Standard 9x9’s in your page because, in the case of Patrick’s, I would suggest to add an specific multiplication factor for the special operations (neg numbers, bitwise OR, mod, exponentiation, … ), empirically obtained from the solutions sent by the puzzlers after some period of time. Finally, instead of giving a decimal point for the rating, perhaps steps of 2 or so should be enough (integer numbers, even, a puzzle with 144 would be just "a little bit" more difficult than another with 142 ).




sneaklyfox
Posted on: Sat Mar 03, 2012 7:10 pm
Posts: 422 Location: Canada Joined: Fri May 13, 2011 2:43 am

Re: what to do with the data?
clm wrote: "the world's population has reached 7 billion persons ... " (translated from English) because for us that is 7.000.000.000.000 and not 7.000.000.000 (here we use the dot instead of the colon) I don't know who uses colon 7:000:000:000:000. : colon Americans use comma 7,000,000,000,000. I use space 7 000 000 000 000. But good to know about the difference in Europe about billion and trillion. People could get confused!




clm
Posted on: Sat Mar 03, 2012 8:38 pm
Posts: 690 Joined: Fri May 13, 2011 6:51 pm

Re: what to do with the data?
sneaklyfox wrote: clm wrote: "the world's population has reached 7 billion persons ... " (translated from English) because for us that is 7.000.000.000.000 and not 7.000.000.000 (here we use the dot instead of the colon) I don't know who uses colon 7:000:000:000:000. : colon Americans use comma 7,000,000,000,000. I use space 7 000 000 000 000. But good to know about the difference in Europe about billion and trillion. People could get confused! Sorry, thank you for the correction, it was my mistake, I should have said comma, my English ...




giulio
Posted on: Sun Mar 04, 2012 2:07 am
Posts: 54 Joined: Thu Nov 03, 2011 8:52 am

Re: what to do with the data?
clm wrote: According to the last Patrick’s clarification the division by the size of the puzzle is not obtained, then only averaging the natural logarithm of the possible permutations of every row and column (after applying the restrictions imposed by each cage). So, for a 9x9:
SR = [ln(P1) + ln(P2) + … + ln(P18)] / 18 if we name P1 … P9 the number of permutations for the rows 1 thru 9 and P10 … P18 for the columns 1 thru 9. This expression is equivalent to: [ln (P1 x P2 x P3 … x P18)] / 18 = natural log of the root of order 18 of the product of all those permutations P1 … P18. Well, that’s a criterion.
In the example you expose, you have 10.1384 as the log for the first row, assuming all 18 rows and columns have a number of permutations in the same range, you would have obtained: SolverRating = 10.2
but not the actual 94.8 assigned to the yesterday’s 9x9 (126 for the february 28’s 9x9, 82.5 for the march 01’s 8x8 and 179.4 for the march 01’s 12x12).
Clearly, in some ways, my calculation is not correct. Even if it reflects my best interpretation of the procedure he describes. I would like to have the formula that, applied to a puzzle, would give me exacly Patrick's rating. Then, I could apply the same formula to my 9x9 puzzles and play with them. For example, when I generate puzzles, I can limit the number of cages that admit more than a given number of combinations. I introduced it because I thought that too many cages with a high number of combinations were going to make the puzzle unsolvable analytically. The puzzles you find in my daily CalcuDoku page ( http://zambon.com.au/puzzles/calcudoku/daily/) are limited to not more than two cages with more than 200 combinations (as you can see in the configuration line immeditely below the puzzle  N_MAX_200). These figures are just picked out of a hat. It woud be nice to see whether and how Patrick's rating would correlate with the number of cages with high numbers of combinations and with the threshold. I was able to solve analytically the first 100 puzzles I generated with max 3 cages containing more than 200 combinations. And then I limited N_MAX_200 to 2 for the online puzzles because I wanted to make them easier. Sometimes, I find Patrick's puzzles too difficult. Useless to say that this can well be due to my limitations (but, as usual, I like to say some useless things ). To put it in a different way: when there are several cages with 1 and the like, I don't manage to reduce the number of combinations. Also most of the puzzles with eight 5cell Wcages in the four corners are, in my opinion, unsolvable analytically. And yet, especially in the case of puzzles with several 1 cages, the number of combinations of each cage are well below 200. This tells me that my criterion is not sufficient to guarantee the generation of analytically solvable puzzles (or, more appropriately said, puzzles that I can solve analytically). clm wrote:  Instead of calculating the possible permutations by rows or columns to do that by “cages” (of two or more cells of course) and averaging by the number of those cages (this must be studied more carefully).
I think we need both. By calculating the number of permutations in rows and columns without considering the cages and then the number of combinations in cages without considering the rows and columns, we might have a good estimate of the complexity of the puzzle. I say 'combinations' in the cages because the number of permutations in a cage, when ignoring rows and columns, is always comb * n!, where 'n' is the size of the cage. It might be appropriate to give more "weight" to larger cages when calculating a rating but, in any case, n! would perhaps make the larger cages count too much. clm wrote:  You might do some day by day analysis with the Patrick’s 5x5’s to correlate the solving rates and adjust the formula just in case you are looking for something similar. Probably the actual ratings in the Patrick’s page are of the type Rating A.
I agree. clm wrote: Anyway, till the moment you are doing Standard 9x9’s in your page because, in the case of Patrick’s, I would suggest to add an specific multiplication factor for the special operations (neg numbers, bitwise OR, mod, exponentiation, … ), empirically obtained from the solutions sent by the puzzlers after some period of time.
Well, I am really not sure about this. I agree that some special operations are more difficult to visualise, but that is only because most of us are unfamiliar with them. I would tend to think that the number of combinations/permutations is what should drive the ratings. On the other hand, if most people find a mod more difficult than a division, perhaps this should be taken into account... Perhaps, Patrick should ask the solvers to rate the difficulty of the puzzles, say, from 1 to 10. Then, it would be fun to see how the official ratings would correlate with the voted ratings. Ultimately, there is no other way to validate the ratings, and this website has enough visitors to achieve a reasonable sample quickly. Without solving the puzzle, people would not be able to vote. To take them into account, Patrick might provide an "I give up" box. Probably many will not like to admit it, but it might still be interesting to have. As I said in several occasions, if I make a mistake in a puzzle, I tend to drop it, rather than restart from scratch. This is because I hate repeating what I have already done, and would have to wait a day or two to feel the puzzle "fresh" again. In those cases, I wouldn't tick the box. But if I really gave up, like it happens with many of the 9x9s with eight 5cell Wcages, I would definitely tick it. Patrick, please add a solverrating field. The sooner we start, the sooner we'll be able to tap the "wisdom of the crowds"!




clm
Posted on: Mon Mar 05, 2012 1:58 pm
Posts: 690 Joined: Fri May 13, 2011 6:51 pm

Re: what to do with the data?
sheldolina wrote: giulio wrote: sneaklyfox wrote: Really??? 10 ^ 12 is a billion in Europe??? 10 ^ 3 thousand 10 ^ 6 ? 10 ^ 9 ? 10 ^ 12 billion 10 ^ 15 trillion??? You have to use languages other than English: 10 ^ 3: Italian=mille; German=Tausend; French=mille 10 ^ 6: milione; Million; million 10 ^ 9: miliardo; Miliarde; milliard 10 ^ 12: bilione; Billion; billion 10 ^ 15: trilione; Trillion; trillion As you can see, miliardo/Miliarde/milliard replace the English billion, and billion and the rest are shifted 'up' one place. And that's still not completely correct  on this Wikipedia page and on this one a full list of respectively French and Dutch names can be found. Congratulations for sheldolina (providing those very interesting references) who being among the younger members of this club (and very brilliant by the way) also has a high skill in English and the scientific curiosity to pay attention to subjects that usually are beyond her age.




clm
Posted on: Mon Mar 05, 2012 2:52 pm
Posts: 690 Joined: Fri May 13, 2011 6:51 pm

Re: what to do with the data?
giulio wrote: ... The puzzles you find in my daily CalcuDoku page ( http://zambon.com.au/puzzles/calcudoku/daily/) are limited to not more than two cages with more than 200 combinations (as you can see in the configuration line immeditely below the puzzle  N_MAX_200). These figures are just picked out of a hat. It woud be nice to see whether and how Patrick's rating would correlate with the number of cages with high numbers of combinations and with the threshold. ... It would be interesting that in a second line you describe a little more those parameters. giulio wrote: ... Sometimes, I find Patrick's puzzles too difficult. Useless to say that this can well be due to my limitations (but, as usual, I like to say some useless things ). To put it in a different way: when there are several cages with 1 and the like, I don't manage to reduce the number of combinations. Also most of the puzzles with eight 5cell Wcages in the four corners are, in my opinion, unsolvable analytically. And yet, especially in the case of puzzles with several 1 cages, the number of combinations of each cage are well below 200. This tells me that my criterion is not sufficient to guarantee the generation of analytically solvable puzzles (or, more appropriately said, puzzles that I can solve analytically). ... However, I think they all can be solved analytically (with minimum trial and error) using simultaneously all techniques (after we have obtained all possible combinations for all cages): addition property of rows and columns, multiplication property, parity, maximums and minimums in the sum of cages, "Xray", common numbers in the intersection areas to make "visible" the "hidden" numbers, possible location of some prime numbers like 5, 7, 11 ... , etc.) but it's true that sometimes they take time (those 9x9's on tuesdays are not very suitable for timed puzzles ) giulio wrote: ... I think we need both. By calculating the number of permutations in rows and columns without considering the cages and then the number of combinations in cages without considering the rows and columns, we might have a good estimate of the complexity of the puzzle. I say 'combinations' in the cages because the number of permutations in a cage, when ignoring rows and columns, is always comb * n!, where 'n' is the size of the cage. It might be appropriate to give more "weight" to larger cages when calculating a rating but, in any case, n! would perhaps make the larger cages count too much. ... This should be carefully analyzed, I mean to assign relative weights to those figures, in case of calculating all the "variations" (combinations and permutations) in the cages if we multiply them we should not need now to do a similar calculation with rows and columns, of course, since the row and column condition facilitates the solution instead of creating more complexity (I would like to know the Patrick's opinion on this). But we could calculate the rows and columns "complexity" and compare. giulio wrote: ... Perhaps, Patrick should ask the solvers to rate the difficulty of the puzzles, say, from 1 to 10. Then, it would be fun to see how the official ratings would correlate with the voted ratings. Ultimately, there is no other way to validate the ratings, and this website has enough visitors to achieve a reasonable sample quickly. ... I had suggested this long time ago: at the moment of sending the "congratulations" message and besides the "OK" to open a small optional window asking (logically only for those who have solved the puzzle) for a number (I had suggested from 1 to 100) to evaluate their "subjective" level of difficulty. giulio wrote: ... In those cases, I wouldn't tick the box. But if I really gave up, like it happens with many of the 9x9s with eight 5cell Wcages, I would definitely tick it. ... Particularly myself I love those tuesday's 9x9s with eight 5cell Wcages, multiplicative, or additive, or both, I find them more challenging than the other, it's my favourite after the 7x7 subtractions only on wednesday's.




beaker
Posted on: Tue Mar 06, 2012 1:20 am
Posts: 490 Location: Ladysmith, BC, Canada Joined: Fri May 13, 2011 1:37 am

Re: what to do with the data?
I envy you clm as I cannot figure out these puzzles....I have solved 2 in the last year....I have spent hours on each of them and not had much success.
Last edited by beaker on Tue Mar 06, 2012 10:20 am, edited 1 time in total.




giulio
Posted on: Tue Mar 06, 2012 4:55 am
Posts: 54 Joined: Thu Nov 03, 2011 8:52 am

Re: what to do with the data?
beaker wrote: I envy you clm as I cannot figure out these puzzles....I have solved 2 in the last year....I have spent hours on each of them and not had much success. Ditto. I joined on 20111103, and I only managed to solve two of those 8W puzzles (on 20111227 and on 20120221). clm wrote: However, I think they all can be solved analytically (with minimum trial and error) using simultaneously all techniques (after we have obtained all possible combinations for all cages): addition property of rows and columns, multiplication property, parity, maximums and minimums in the sum of cages, "Xray", common numbers in the intersection areas to make "visible" the "hidden" numbers, possible location of some prime numbers like 5, 7, 11 ... , etc.) but it's true that sometimes they take time (those 9x9's on tuesdays are not very suitable for timed puzzles ) My apologies for being picky, but if you say "with minimum trial and error", it means that they are not solved analytically. I consider a puzzle to be analytically solvable only when I am certain of all numbers that I write in them. Even if sometimes, to do so, I need to make lists and consider how permutations of different cages eliminate each other. And even if, sometimes, my certainties are misplaced, I make mistakes, and have to start from scratch. clm wrote: giulio wrote: I think we need both. By calculating the number of permutations in rows and columns without considering the cages and then the number of combinations in cages without considering the rows and columns, we might have a good estimate of the complexity of the puzzle. I say 'combinations' in the cages because the number of permutations in a cage, when ignoring rows and columns, is always comb * n!, where 'n' is the size of the cage. It might be appropriate to give more "weight" to larger cages when calculating a rating but, in any case, n! would perhaps make the larger cages count too much. This should be carefully analyzed, I mean to assign relative weights to those figures, in case of calculating all the "variations" (combinations and permutations) in the cages if we multiply them we should not need now to do a similar calculation with rows and columns, of course, since the row and column condition facilitates the solution instead of creating more complexity (I would like to know the Patrick's opinion on this). But we could calculate the rows and columns "complexity" and compare. It seems to me that, if we consider the permutations within rows and columns without considering the cages, we should also consider the combinations of the cages without considering rows and columns, and then combine the two.




beaker
Posted on: Tue Mar 06, 2012 10:25 am
Posts: 490 Location: Ladysmith, BC, Canada Joined: Fri May 13, 2011 1:37 am

Re: what to do with the data?
Alas, I have just figured out my 3rd Tuesday 9x9 in about 1.5 hours.....now if I can only figure out why the last number entered in the "bit/wise puzzle" was wrong (only 2nd time I've tried one of these using clm's tables of numbers posted Oct 18/2011)




clm
Posted on: Tue Mar 06, 2012 1:47 pm
Posts: 690 Joined: Fri May 13, 2011 6:51 pm

Re: what to do with the data?
beaker wrote: I envy you clm as I cannot figure out these puzzles....I have solved 2 in the last year....I have spent hours on each of them and not had much success. Any small mistake when breaking into prime factors, or when obtaining all possible combinations for the cages "x", ":", "", ... , makes the full puzzle to crash earlier or later. So the first steps are very important, and in this phase, do not be affraid of loosing some time. Later, of course, we must also be careful (very sure) when erasing any combination. I am sure in a short time you will be solving all. It is true that among the 9x9's on tuesdays there are some of them more difficult than other. In fact I would like to know what would be the "solver rating" for the most difficult that could be dessigned by the Patrick's software (with the unique solution as usual) (a different situation would be to know the upper limit of the "solver rating", the maximum, for any of the actual puzzles). And I would suggest Patrick to provide that 9x9 apart, in a post, just to see if we can solve it (perhaps in the Section "Specific puzzles / your own puzzles") and I would affirm in advance that there are puzzlers in the site (I do not say I could do it) who would get the solution, exactly as with the very few "almost impossible" in the "101 Advanced Puzzles" book.






You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum

