Utah Open Source Planet

Any other cyclists out there? You a chainlove junkie too? After finding myself spending too much time refreshing and refreshing, I wrote the following script to watch the site for things I was interested in and send me a text message when they come up. I put it on my machine that’s always on and scheduled it to run every ten minutes with launchd. Hopefully it will save you a little time and sanity as well. (Note the wrapped lines.)

/* 
 * CHAINLOVE ALERTS
 * 
 * This script scrapes chainlove.com and sends an email when words in 
 * the "watch list" are included in the current product offering.
 */
 
// list of words to watch for
$watch_list = array('sunglasses', 'warmers', 'jersey', 'giordana');
 
// cache file (including path)
$cache_file = '/tmp/chainlove_alerts.cache';
 
// scrape page
$ch = curl_init() or die(curl_error()); 
curl_setopt($ch, CURLOPT_URL, "http://www.chainlove.com"); 
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
$page = curl_exec($ch) or die(curl_error()); 
curl_close($ch);
 
// pull out the product details
preg_match('/<h1 id="item_title">([^<]+)<\/h1>.*<h3 class="price"> ¬
	([^<]+)<\/h3>.*<div id="regular_price">\n*\t*\s*([^<]+)<br \/> ¬
	\n\t*\s*([^<]+)<\/div>/sm', $page, $matches);
 
// compare current product against cache product... if same, exit
if(!file_exists($cache_file)) {
	file_put_contents($cache_file, trim($matches[1]));
} else {
	$cache_product = file_get_contents($cache_file);
 
	if($cache_product == trim($matches[1])) {
		exit();
	} else {
		file_put_contents($cache_file, trim($matches[1]));
	}
}
 
// search current product title with "watch list"
$send_mail = 0;
foreach($watch_list as $watch_item) {
	if(stripos($matches[1], $watch_item)) {
		$send_mail++;
	}
}
 
// send email
if($send_mail > 0) {
	$mail['to'] = '8015551212@txt.att.net';
	$mail['subject'] = trim($matches[1]);
	$mail['message'] = trim($matches[2]) . ' (' . trim($matches[3]) .
		', ' . trim($matches[4]) . ')';
	$mail['from'] = 'Chainlove Alerts <user@example.com>';
	$mail['additional_headers'] = 'From: ' . $mail['from'];
	mail($mail['to'], $mail['subject'], $mail['message'], ¬
		$mail['additional_headers']);
}

I had a friend send me a list of these questions today. some are repeats from previous posts but there are several new ones. Enjoy!

Logic Problems

a) 8 Steel balls, 1 heavier than all the rest. Use a Balance Scale to determine the heaviest ball in the shortest number of tries.

b) Find how many trailing zeroes are in the product of 100! (100 Factorial).

c) Replace each letter below with a corresponding number from 0 to 8. Each Letter can only be matched to one number (ie, all of the E’s would be a 3, and that would mean that no other Letter could be a 3). The following addition problem should work after you replace the letters.

   S  E  V  E  N
   S  E  V  E  N
+         S   I   X
-----------------------
 T W E N  T   Y

Programming Problems

a) Write a function that takes an array of integers from 1 to 1 million, with one number being duplicated. Determine which integer is a duplicate, accounting for speed and efficiency.

b) Write a function that takes an array of integers from 1 to 1 million, with one number being missing from the range. Determine the missing integer, account for speed and efficiency.

c) Write a function that takes an array of an number of integers. Determine the largest contiguous sum, meaning given any two integers in the array, what is the highest sum? Given an array such as Array(1, -5, 3, -3, -9, 2) the highest contiguous sum would be 5.

d) Write a function that takes a string as a parameter and determine the first unique character in the string. In other words, given the string “sissy” the answer would be “i”. Afterwards, how would you optimize the function.

e) Pseudo-code the Ball Clock problem, determining how many cycles it takes for the balls to end up in the same position in the queue tray. Afterwards, how would you optimize the process if it were to be hit thousands of times on a given interface by thousands of users. If you have come up with a pure brute-force method, how could you optimize it further to shorten the processing time?

reversing a string is a classic tech interview question. of course you’re not allowed to call any built-in reverse() function. A solution might look like, in my pseudocode:

someString = new string("732432424");
char temp;
int end=someString.length()-1;

for (int x=0;x<end;x++)
{
	temp=someString[end];
	someString[x]=someString[end];
	someString[end]=temp;
	end--;
}

but sometimes the interviewer asks you to reverse the string in place, without using other memory, and for that, we’ll need our XOR friend

A B  XOR
----+---
0 0 | 0
0 1 | 1
1 0 | 1
1 1 | 0

someString = new string("732432424");
char temp;
int end=someString.length()-1; //assume zero based char arrays

for (int x=0;x<end;x++)
{
	someString[x]=someString[x] XOR someString[end];
        // first loop: someString[x]=3
	someString[end]=someString[end] XOR someString[x];
        //first loop: someString[end]=7
	someString[x] = someString[x] XOR someString[end];
       //first loop: someString[x]=4
	end--;
}

I've released yet again. Go to the web page for the details. Now Crème Rappel has its own RSS feed so I'll shut up about here now.

1. http://www.php.net/manual/en/control-structures.alternative-syntax.php shows an alternative syntax for loops:

    foreach($list as $item)
    {
        echo $item;
    } 

can be done as:

    foreach($list as $item):
        echo $item;
    endforeach

2. if your server has short tags enabled, you can do this:

<?

$somevar="hello";

?>

<?=$somevar?>

this script checks to see if a program is running and starts it if it is not. if a certain time has passed the script will kill the running program

#!/bin/bash
 
#location of script to monitor
PROGRAM=‘myscript.php’
COUNTER=0
 
#!/bin/bash
 
date2stamp () {
    date –utc –date "$1" +%s
}
 
# convert a date into a UNIX timestamp
# time afterwhich script should not run
finishtime=$(date2stamp "Thu Nov  1 15:22:00 MDT 2007")
#echo $finishtime
 
while [ 1 ];
do
 
nowtime=`date –utc +%s`
#echo $nowtime
diffTime=$((finishtime-nowtime))
if ((diffTime > 0));
then
echo "still time to go"
else
echo "TIME UP!: checking if running:"
PROG_CHECK=`ps aux|grep $PROGRAM|wc -l`
if [ $PROG_CHECK -gt 1 ];
then
echo "running after time up"
PROG_ID=`ps uax|grep $PROGRAM |head -n 1|awk ‘{print $2}’`
#echo "KILLING"
#kill -9 $PROG_ID
else
echo "not running after time up"
fi
exit 65
fi
 
 
PROG_CHECK=`ps aux|grep $PROGRAM|wc -l`
if [ $PROG_CHECK -gt 1 ];
then
echo "running"
else
let COUNTER=COUNTER+1
echo "not running: starting: new log file: $COUNTER"
/usr/local/bin/php /root/$PROGRAM &
fi
 
#time in seconds to sleep
sleep 5
 
done

We recently upgraded from MySQL 4 to MySQL 5 and we noticed strange characters in some of the varchar and text fields.

step 1: dump out the contents of the bad field in hex
mysql> select hex(myfieldname) from mytablename where myid=’something’;

step 2: run a php program to print out the chars one at a time

function hex2asc($temp) {
  $len = strlen($temp);
  for ($i=0;$i<$len;$i+=2) {
  $data[$i]=chr(hexdec(substr($temp,$i,2)))."  ".substr($temp,$i,2);
 
  }
  return $data;
}
$str="HEX GOES HERE";
$data=hex2asc($str);
print_r($data);

step 3: once you’ve identified the bad chars and their hex values, change them in the database

here are some bad chars I found:

– fix apostrophe
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’92′),”‘”);

– fix left single quote
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’93′),”‘”);

– fix right single quote
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’94′),”‘”);

– fix bullets
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’95′),”& #8226;”);

– fix double dash
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’96′),”–”);

– fix triple dash
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’97′),”—”);

— fix supscripted TM
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’99′),”^TM”);

– replace jacked up apostrophe/single quote
– delete EFs
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’EF’),”");
– replace BFs with single quote
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’BF’),”‘”);
– delete BDs
update mytablename set myfieldname=REPLACE(myfieldname,UNHEX(’BD’),”");

see also: http://www.oreillynet.com/onlamp/blog/2006/01/turning_mysql_data_in_latin1_t.html

Real Age calculators have become all the rage lately and so I decided to reverse engineer (and improve) a popular one (http://www.poodwaddle.com/realage.htm .) The poodwaddle calc is made in flash, but I made mine in php and javascript with an XML backend. Below is the code I used to read the Real Age XML into a PHP array. (oh, and you’ll also need the PHP XML parsing class from php.net.)

1. <?
2. if (!isset($_SESSION['questionArray'])) //cache
3. {
4. $lookupTable=array();
5. foreach($xml->children[0]->children as $lookup)
6. {
7. if ($lookup->name=='row')
8. $lookupTable[$lookup->attributes["input"]]=$lookup->attributes["output"];
9. }
10.  
11. $questionArray=array();
12. foreach($xml->children[1]->children as $questions)
13. {
14. $tempQuestion=new Question();
15. $tempQuestion->title=$questions->children[0]->content;
16. $tempQuestion->prompt=$questions->children[1]->content;
17. $tempQuestion->genderSpecific=$questions->children[2]->content;
18. $tempQuestion->controllable=$questions->children[4]->content;
19. $tempQuestion->options=array();
20. foreach($questions->children[3]->children as $option)
21. {
22. $optionPrompt=$option->attributes["prompt"];
23. $RAEffect=$option->attributes["RA-effect"];
24. $tempQuestion->options[$optionPrompt]=$RAEffect;
25. }
26. $questionArray[]=$tempQuestion;
27. }
28. $_SESSION['questionArray']=$questionArray;
29. $_SESSION['lookupTable']=$lookupTable;
30. }
31. ?>
32. Download this code: parseData.inc.phps

The actual Real Age code is pretty ugly, but I’ll post it as soon as I clean it up. Yes it works in IE and FF. Yes, the javascript slider code is slow (but it works!). It’s from the dojokit. It was the only vertical slidebar that allowed custom labels I could find.

I recently stumbled upon Javascript/Canvas Graph library which allows you to create cool network graphs with nodes and interconnections. Now that I had a graphic library, I needed something with lots of connections to graph. That’s when I thought of mapping health IT blog sites. Now all I needed was a spider…

Spiders, robots, crawlers all work on the same principle: start with a seed url(s), request the urls and then parse out the resultant html for all link tags which are then pushed into the queue for crawling.

time for some code:

1. <?
2. $tldList=file("tlds.txt");
3. $masterList=file("masterList.txt");
4. $acceptableNodes=array();
5. foreach($masterList as $node)
6. {
7. $node="http://".trim($node);
8. $acceptableNodes[]=parse_url_domain ($node);
9. }
10. //print_r($acceptableNodes);
11. $connections=array();
12. $nodeCounter=array();
13. foreach ($masterList as $webpage)
14. {
15. $url ="http://".trim($webpage);
16. $strippedWebpage=parse_url_domain ($url);
17. $input = file_get_contents($url); //or die("Could not access file: $url");
18. if (!$input) continue;
19. $regexp = "<a\s[^>]*href=(\"??)([^\" >]*?)\\1[^>]*>(.*)<\/a>";
20. $URLs=array();
21. if(preg_match_all("/$regexp/siU", $input, $matches, PREG_SET_ORDER))
22. {
23. foreach($matches as $match)
24. {
25. //echo($match[2]."\t");
26. $currentURL=parse_url_domain ($match[2]);
27. //echo($currentURL);
28. //echo("\n");
29. if ($currentURL!=$webpage && !in_array($currentURL,$URLs)&&check_domain($currentURL)&&check_tld($currentURL))
30. {
31. $URLs[]=$currentURL;
32. if (in_array($currentURL,$acceptableNodes)&& ($strippedWebpage!=$currentURL))
33. {
34. $connections[]="$strippedWebpage -> $currentURL";
35. $nodeCounter[$currentURL]++;
36. }
37. }
38.  
39. }
40. }
41. echo("$webpage : \n");
42. print_r($URLs);
43. }
44. echo("connections: ");
45. print_r($connections);
46. echo("nodeCounter: ");
47. print_r($nodeCounter);
48. echo("javascript edges: ");
49. $multiplier=3;
50. foreach($connections as $connection)
51. {
52. list($from,$to)=explode(" -> ",$connection);
53. $fromCtr=3*$nodeCounter[$from];
54. $toCtr=3*$nodeCounter[$to];
55. echo("g.addEdge($('".$from."'), $('".$to."'),".$fromCtr.",".$toCtr.");\n");
56. }
57. function parse_url_domain ($url)
58. {
59.  
60. $raw_url= parse_url($url);
61. if ($raw_url['host'] == '')
62. {
63. $raw_url['host'] = $raw_url['path'];
64. }
65. $domain_only[1] = $raw_url['host'];
66. return strtolower($domain_only[1]);
67. }
68. function check_domain ($url)
69. {
70. if (!ereg("^.*\..*$", $url))
71. {
72. return false;
73. }
74. $local_array = explode(".", $url);
75. for ($i = 0; $i < sizeof($local_array); $i++)
76. {
77. if (!ereg("^(([A-Za-z0-9!#$%&'*+/=?^_`{|}~-][A-Za-z0-9!#$%&'*+/=?^_`{|}~\.-]{0,63})|(\"[^(\\|\")]{0,62}\"))$", $local_array[$i]))
78. {
79. return false;
80. }
81. }
82. return true;
83. }
84. function check_tld($url)
85. {
86. global $tldList;
87. $parts=explode(".",$url);
88. $lastpart=trim($parts[count($parts)-1]);
89. foreach($tldList as $item)
90. {
91. if (trim($item)==$lastpart)
92. return true;
93. }
94. return false;
95.  
96. }
97. ?>
98. Download this code: linkSpider.phps

and no, I’m not talking about the Utah Summer Games Arm Wrestling competition.

I’m talking about Python, the programming language. And to encourage adoption of that language some clever people created a little Python programming game (back in 2005.) You’re presented with a riddle of sorts and you’re supposed to code up a python script to solve it. Only, I don’t like people telling me what to do, so I intentionally did NOT code up python to solve them. I used PHP mostly. Here are my notes:

QUESTION 1: what’s 2^38?
ANSWER: I asked google: http://www.google.com/search?hl=en&q=2%5E38&btnG=Google+Search
and then I used the answer to get to the next step: 274877906944.html

QUESTION 2: given: K->M, O->Q, E->G, decode: g fmnc wms bgblr rpylqjyrc gr zw fylb. rfyrq ufyr amknsrcpq ypc dmp. bmgle gr gl zw fylb gq glcddgagclr ylb rfyr’q ufw rfgq rcvr gq qm jmle. sqgle qrpgle.kyicrpylq() gq pcamkkclbcb. lmu ynnjw ml rfc spj.
ANSWER: I used an online cryptogram solver (though i could have written a simple script to shift up two letters…)
“map” then, shifts to “ocr”, which leads to http://www.pythonchallenge.com/pc/def/ocr.html

QUESTION 3: Find rare characters in the mess below (mess omitted)
ANSWER: php:
$challenge=file_get_contents('phychall2.txt');
echo(ereg_replace("[^a-z]","",$challenge));

which produces: http://www.pythonchallenge.com/pc/def/equality.html

QUESTION 4: “One small letter, surrounded by EXACTLY three big bodyguards on each of its sides.”
ANSWER: I used textpad’s regex ([a-z][A-Z][A-Z][A-Z][a-z][A-Z][A-Z][A-Z][a-z]) and found “linkedlist”, which means: http://www.pythonchallenge.com/pc/def/linkedlist.html

QUESTION 5: “urllib may help. DON’T TRY ALL NOTHINGS, since it will never end. 400 times is more than enough”
ANSWER:
$linked="12345";
for($x=0;$x<400;$x++)
{
$page=file_get_contents("http://www.pythonchallenge.com/pc/def/linkedlist.php?nothing=$linked");
echo($page."\n");
$previouslinked=$linked;
$linked='';
list($blank,$linked)=@split('and the next nothing is ',$page);
$pos = strpos($page, "Divide");
if ($pos === false)
{
if ($linked=='') die();
}
else
{
$linked=$previouslinked/2;
echo("divide found!!");
}
echo($linked."\n");
}

which produces: http://www.pythonchallenge.com/pc/def/peak.html

QUESTION 6: peak hell
ANSWER:
arghh. have to use python…need to load the pickle dump:

import pickle
unpicklefile = open('banner.p', 'r')
unpickledlist = pickle.load(unpicklefile)
unpicklefile.close()
for item in unpickledlist:
print item

python unpickle.py > nextstep.txt

lines look like: [(’ ‘, 14), (’#', 5), (’ ‘, 70), (’#', 5), (’ ‘, 1)]

parse them with:

$problem=file("nextstep.txt");
$parseout=array('(',']',')','[',"'");
function printit($value,$times)
{
for($loop=0;$loop<$times;$loop++)
echo($value);
}
foreach($problem as $line)
{
$tuples=split('),',$line);
foreach($tuples as $tuple)
{
list($first,$second)=split(',',$tuple);
$first=trim(str_replace($parseout,'',$first));
if ($first=='') $first=' ';
$second=trim(str_replace($parseout,'',$second));
printit($first, $second);
}
echo("\n");
}


which produces a nifty ascii art version of: http://www.pythonchallenge.com/pc/def/channel.html

QUESTION 7: : now there are pairs (in http://www.pythonchallenge.com/pc/def/channel.zip)
ANSWER:

$linked="90052";
for($x=0;$x<909;$x++)
{
$bigArray[]=system("zipinfo -v channel.zip {$linked}.txt|tail -3|head -1",$retval);
$page=file_get_contents("{$linked}.txt");
$previouslinked=$linked;
$linked='';
list($blank,$linked)=@split('Next nothing is ',$page);
$pos = strpos($page, "Collect");
if ($pos === false);
else
{
$x=10000;
}
}
$counter=2;
foreach($bigArray as $dot)
{
if ($dot=='') $dot=' ';
echo($dot);
if ($counter++%65==1) echo("\n");
}

Which produces more ASCII art and leads to: http://www.pythonchallenge.com/pc/def/oxygen.html

QUESTION 8: the next level is [105, 110, 116, 101, 103, 114, 105, 116, 121]
ANSWER:

$im = imagecreatefrompng("oxygen.png");
for($x=3;$x<629;$x+=7)
{
$rgb = imagecolorat($im, $x, 46);
$r = ($rgb >> 16) & 0xFF;
$g = ($rgb >> & 0xFF;
$b = $rgb & 0xFF;
echo(chr($r));
}

more code:

$thelist=array(105, 110, 116, 101, 103, 114, 105, 116, 121);
foreach($thelist as $entry)
{
echo(chr($entry));
}


http://www.pythonchallenge.com/pc/def/integrity.html

QUESTION 9:
un: ‘BZh91AY&SYA\xaf\x82\r\x00\x00\x01\x01\x80\x02\xc0\x02\x00 \x00!\x9ah3M\x07<]\xc9\x14\xe1BA\x06\xbe\x084'
pw: 'BZh91AY&SY\x94$|\x0e\x00\x00\x00\x81\x00\x03$ \x00!\x9ah3M\x13<]\xc9\x14\xe1BBP\x91\xf08'
ANSWER: missing link : bee
use commandline python, arghh…

s = "BZh91AY&SYA\xaf\x82\r\x00\x00\x01\x01\x80\x02\xc0\x02\x00 \x00!\x9ah3M\x07<]\xc9\x14\xe1BA\x06\xbe\x084"
import bz2
bz2.decompress(s)
'huge'
t = "BZh91AY&SY\x94$|\x0e\x00\x00\x00\x81\x00\x03$ \x00!\x9ah3M\x13<]\xc9\x14\xe1BBP\x91\xf08"
bz2.decompress(t)
'file'