Browsing the tag ruby
Ruby LocaleTranslator
Leave a comment | Filed under main open source programming
(2 votes, average: 3.50 out of 5)
Loading ...Today, I am open sourcing my Ruby LocaleTranslator; the translator uses google’s translator API to translate a primary seed locale into various other languages. This eases the creation of multi-lingual sites. Not only can the LocaleTranslator translate your main seed locale into different languages but it can also recursively merge in differences, this comes in handy if you have hand-optimized your translated locales.
Viva Localization!
My projects that use the LocaleTranslator; UploadBooth, PasteBooth and ShrinkBooth.
LocaleTranslator Examples
en.yml
site: hello_world: Hello World! home: Home statement: Localization should be simple!
Batch Conversion of your English locale.
require ‘monkey-patches.rb’
require ‘locale_translator.rb’
en_yml = YAML::load(File.open(‘en.yml’))
[:de,:ru].each do |lang|
lang_yml = LocaleTranslator.translate(en_yml,
:to=>lang,
:html=>true,
:key=>‘GOOGLE API KEY’)
f = File.new("#{lang.to_s.downcase}.yml","w")
f.puts(lang_yml.ya2yaml(:syck_compatible => true))
f.close
p "Translated to #{lang.to_s}"
end
Merge in new locale keys from your English Locale into your already translated Russian locale.
require ‘monkey-patches.rb’
require ‘locale_translator.rb’
en_yml = YAML::load(File.open(‘en.yml’))
ru_yml = YAML::load(File.open(‘ru.yml’))
ru_new_yml = LocaleTranslator.translate(en_yml,
:to=>:ru,
:html=>true,
:merge=>ru_yml,
:key=>‘GOOGLE API KEY’)
puts ru_new_yml.ya2yaml(:syck_compatible => true)
The Implementation Code
Support Monkey Patches
monkey-patches.rb
def to_list
h2l(self)
end
def diff(hash)
hsh = {}
this = self
hash.each do |k,v|
if v.kind_of?Hash and this.key?k
tmp = this[k].diff(v)
hsh[k] = tmp if tmp.size > 0
else
hsh[k] = v unless this.key?k
end
end
hsh
end
def merge_r(hash)
hsh = {}
this = self
hash.each do |k,v|
if v.kind_of?Hash
hsh[k] = this[k].merge_r(v)
else
hsh[k] = v
end
end
self.merge(hsh)
end
private
def h2l(hash)
list = []
hash.each {|k,v| list = (v.kind_of?Hash) ? list.merge_with_dups(h2l(v)) : list << v }
list
end
end
class Array
def chunk(p=2)
return [] if p.zero?
p_size = (length.to_f / p).ceil
[first(p_size), *last(length - p_size).chunk(p - 1)]
end
def to_hash(hash)
l2h(hash,self)
end
def merge(arr)
self | arr
end
def merge_with_dups(arr)
temp = []
self.each {|a| temp << a }
arr.each {|a| temp << a }
temp
end
def merge!(arr)
temp = self.clone
self.clear
temp.each {|a| self << a }
arr.each {|a| self << a unless temp.include?a }
true
end
def merge_with_dups!(arr)
temp = self.clone
self.clear
temp.each {|a| self << a }
arr.each {|a| self << a }
true
end
private
def l2h(hash,lst)
hsh = {}
hash.each {|k,v| hsh[k] = (v.kind_of?Hash) ? l2h(v,lst) : lst.shift }
hsh
end
end
The LocaleTranslator Implementation
You need the ya2yaml and easy_translate gems. Ya2YAML can export locales in UTF-8 unlike the standard yaml implementation that can only export in binary for non-standard ascii.
locale-translator.rb
require ‘rubygems’
require ‘ya2yaml’
require ‘yaml’
require ‘easy_translate’
class LocaleTranslator
def self.translate(text,opts)
opts[:to] = [opts[:to]] if opts[:to] and !opts[:to].kind_of?Array
if opts[:merge].kind_of?Hash and text.kind_of?Hash
diff = opts[:merge].diff(text)
diff_hsh = LocaleTranslator.translate(diff,:to=>opts[:to],:html=>true)
return opts[:merge].merge_r(diff_hsh)
end
if text.kind_of?Hash
t_arr = text.to_list
t_arr = t_arr.first if t_arr.size == 1
tout_arr = LocaleTranslator.translate(t_arr,:to=>opts[:to],:html=>true)
tout_arr = [tout_arr] if tout_arr.kind_of?String
tout_arr.to_hash(text)
elsif text.kind_of?Array
if text.size > 50
out = []
text.chunk.each {|l| out.merge_with_dups!(EasyTranslate.translate(l,opts).first) }
out
else
text = text.first if text.size == 1
EasyTranslate.translate(text,opts).first
end
else
EasyTranslate.translate(text,opts).first
end
end
end
Ruby ThreadSafe JSMin Library
1 Comment | Filed under main (1 votes, average: 5.00 out of 5) Loading ...I needed a thread-safe JSMin library for compressing javascripts on the fly on UploadBooth, so I took an existing ruby implementation and made it thread safe. I don’t think there was license defined when I got it, so I am re-releasing it as-is.
class JSMin
EOF = -1
include MonitorMixin
# jsmin — Copy the input to the output, deleting the characters which are
# insignificant to JavaScript. Comments will be removed. Tabs will be
# replaced with spaces. Carriage returns will be replaced with linefeeds.
# Most spaces and linefeeds will be removed.
# thread safe
def minimize(jstext)
synchronize do
@theA = ""
@theB = ""
@current = 0
@output = ""
@text = jstext
@theA = "\n"
action(3)
while (@theA != JSMin::EOF)
case @theA
when " "
if (isAlphanum(@theB))
action(1)
else
action(2)
end
when "\n"
case (@theB)
when "{","[","(","+","-"
action(1)
when " "
action(3)
else
if (isAlphanum(@theB))
action(1)
else
action(2)
end
end
else
case (@theB)
when " "
if (isAlphanum(@theA))
action(1)
else
action(3)
end
when "\n"
case (@theA)
when "}","]",")","+","-","\"","\\", "’", ‘"’
action(1)
else
if (isAlphanum(@theA))
action(1)
else
action(3)
end
end
else
action(1)
end
end
end
@output
end
end
private
# isAlphanum — return true if the character is a letter, digit, underscore,
# dollar sign, or non-ASCII character
def isAlphanum(c)
return false if !c || c == JSMin::EOF
return ((c >= ‘a’ && c <= ‘z’) || (c >= ’0′ && c <= ’9′) ||
(c >= ‘A’ && c <= ‘Z’) || c == ‘_’ || c == ‘$’ ||
c == ‘\’ || c[0] > 126)
end
# get — return the next character from stdin. Watch out for lookahead. If
# the character is a control character, translate it to a space or linefeed.
# thread safe
def get
return JSMin::EOF if @current>(@text.length-1)
c = @text[@current]
@current += 1
c = c.chr
return c if (c >= " " || c == "\n" || c.unpack("c") == JSMin::EOF)
return "\n" if (c == "\r")
return " "
end
# Get the next character without getting it.
def peek
lookaheadChar = @text[@current]
return lookaheadChar.chr
end
# mynext — get the next character, excluding comments.
# peek() is used to see if a ‘/‘ is followed by a ‘/‘ or ‘*‘.
def mynext
c = get
if (c == "/")
if(peek == "/")
while(true)
c = get
if (c <= "\n")
return c
end
end
end
if(peek == "*")
get
while(true)
case get
when "*"
if (peek == "/")
get
return " "
end
when JSMin::EOF
raise "Unterminated comment"
end
end
end
end
return c
end
# action — do something! What you do is determined by the argument: 1
# Output A. Copy B to A. Get the next B. 2 Copy B to A. Get the next B.
# (Delete A). 3 Get the next B. (Delete B). action treats a string as a
# single character. Wow! action recognizes a regular expression if it is
# preceded by ( or , or =.
def action(a)
if(a==1)
@output << @theA
end
if(a==1 || a==2)
@theA = @theB
if (@theA == "\’" || @theA == "\"")
while (true)
@output << @theA
@theA = get
break if (@theA == @theB)
raise "Unterminated string literal" if (@theA <= "\n")
if (@theA == "\\")
@output << @theA
@theA = get
end
end
end
end
if(a==1 || a==2 || a==3)
@theB = mynext
if (@theB == "/" && (@theA == "(" || @theA == "," || @theA == "=" ||
@theA == ":" || @theA == "[" || @theA == "!" ||
@theA == "&" || @theA == "|" || @theA == "?" ||
@theA == "{" || @theA == "}" || @theA == ";" ||
@theA == "\n"))
@output << @theA
@output << @theB
while (true)
@theA = get
if (@theA == "/")
break
elsif (@theA == "\\")
@output << @theA
@theA = get
elsif (@theA <= "\n")
raise "Unterminated RegExp Literal" + @output
end
@output << @theA
end
@theB = mynext
end
end
end
end
Nginx Passenger vs JRuby on Jetty
3 Comments | Filed under administration benchmark (2 votes, average: 4.00 out of 5) Loading ...I am in the process of evaluating which option to choose for a new production deployment of a Sinatra application.
Pros and Cons of the implementations:
JRuby Stack:
Pros:
• Multi-threaded, easy to scale with spiked traffic / shared resources.
Cons:
• Single process is a single point of failure.
MRI Ruby Stack:
Pros:
• Scaled via processes, no single point of failure.
Cons:
• Single Process, no shared resources (Possibly using more memory over time).
These tests are run against a real-world application that is soon to be released, not some dummy “hello world” application.
Application Background:
Sinatra / HAML templates (not compiled, rendered per request) / CouchDB / R18N Translation
Server Specifications:
Hardware: 8Gig / Quad Core Xeon x5355
MRI Stack:
Ruby 1.8.7 (2008-08-11 patchlevel 72)
Nginx Passenger 2.2.4
Passenger Config: passenger_max_pool_size 8, passenger_use_global_queue on
Java Stack:
JRuby 1.3.1 (ruby 1.8.6p287) (2009-06-15 6586)
Jetty-6.1.15
JDK Flags: -server -Xverify:none -XX:MaxPermSize=96m -XX:+AggressiveOpts -Xss128k -Xms256m -Xmx384m -XX:+UseParallelGC -XX:+UseParallelOldGC
JDK 1.7.0 b67
Here are the results. I have taken the best time out of 10 runs, giving enough time for the JDK to warmup and passenger to load all the children. The results are clipped for brevity.
Benchmark command:
JRuby Results:
Time per request: 116.316 [ms] (mean)
Time taken for tests: 11.632 seconds
Memory Use After Test: 437M (RSS)
MRI Results:
Time per request: 84.142 [ms] (mean)
Time taken for tests: 8.414 seconds
Memory Use After Test: 264M (RSS)
Conclusions and final thoughts:
Seems like MRI Ruby has a 39% performance advantage on JRuby executing my application. I am still a bit skeptical if MRI Ruby would still win out in production when it turns into a long running process marathon with varied traffic patterns. At the end of the day the JVM currently has the edge in garbage collection on MRI Ruby, so in “theory” JRuby should be the better choice. This is all a hypothetical guesstimate[sic] on my behalf. I will most likely end up trying both variants in production and see which works best.
Catalyst vs Rails Round Two
11 Comments | Filed under administration benchmark (6 votes, average: 3.17 out of 5) Loading ...There seems to be some interest Catalyst vs Rails vs Django benchmark. The older benchmark is quite old, it was done in 2007. A lot has changed since then. I am re-running the numbers once again to see what has changed. This time around the hardware is faster and the benchmark is slightly more simple. I am just stress testing the controller response performance of the two frameworks.
Benchmark System:
Quad Core Xeon x5355 @ 2.66GHz,8 Gigs Ram,OpenSolaris SNV98
Quick Summary:
Catalyst 5.8/Perl 5.10: 611.78req/sec (Single Process,bsdmalloc)
Catalyst 5.8/Perl 5.10: 1485.53req/sec (Multi Process,bsdmalloc)
Rails 2.3.2/MRI Ruby 1.8.7: 259.93req/sec (Single Process,bsdmalloc)
Rails 2.3.2/JRuby 1.3-dev: 311.71req/sec (Single-Threaded,bsdmalloc)
Rails 2.3.2/JRuby 1.3-dev: 992.32req/sec (Multi-Threaded,libumem)
Rails 2.3.2/MRI Ruby 1.9.1: 603.92req/sec (Single Process,bsdmalloc)
Catalyst 5.8 / Perl 5.10
Compiled: SUNCC -xO5 -xipo -fast -xtarget=native
# ab -n1000 -c100 http://somedomain:3000/ This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Copyright 2006 The Apache Software Foundation, http://www.apache.org/ Benchmarking localhost (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Finished 1000 requests Server Software: Server Hostname: somedomain Server Port: 3000 Document Path: / Document Length: 11 bytes Concurrency Level: 100 Time taken for tests: 0.673159 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 159000 bytes HTML transferred: 11000 bytes Requests per second: 1485.53 [#/sec] (mean) Time per request: 67.316 [ms] (mean) Time per request: 0.673 [ms] (mean, across all concurrent requests) Transfer rate: 230.26 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 2.3 0 35 Processing: 15 62 10.8 62 103 Waiting: 15 59 11.6 61 98 Total: 15 62 10.6 63 103 Percentage of the requests served within a certain time (ms) 50% 63 66% 66 75% 69 80% 71 90% 74 95% 76 98% 86 99% 91 100% 103 (longest request)
Rails 2.3.2 / Ruby 1.8.7
Compiled: SUNCC -xO5 -xipo -fast -xtarget=native
# ab -n1000 -c100 http://somedomain:3000/main/index This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking somedomain (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Mongrel Server Hostname: somedomain Server Port: 3000 Document Path: /main/index Document Length: 11 bytes Concurrency Level: 100 Time taken for tests: 3.847 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 290003 bytes HTML transferred: 11000 bytes Requests per second: 259.93 [#/sec] (mean) Time per request: 384.718 [ms] (mean) Time per request: 3.847 [ms] (mean, across all concurrent requests) Transfer rate: 73.61 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.1 0 16 Processing: 10 369 65.2 389 428 Waiting: 9 368 65.3 388 427 Total: 10 369 65.2 390 428 Percentage of the requests served within a certain time (ms) 50% 390 66% 396 75% 398 80% 400 90% 404 95% 407 98% 413 99% 417 100% 428 (longest request)
Rails 2.3.2 / JRuby 1.3-dev build 6586 (Multi-Threaded), libumem
Platform: JDK7 B56
# ab -n1000 -c100 http://somedomain.com:3000/main/index
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking somedomain.com (be patient)
Completed 100 requests
Completed 200 requests
Completed 300 requests
Completed 400 requests
Completed 500 requests
Completed 600 requests
Completed 700 requests
Completed 800 requests
Completed 900 requests
Completed 1000 requests
Finished 1000 requests
Server Software:
Server Hostname: somedomain.com
Server Port: 3000
Document Path: /main/index
Document Length: 11 bytes
Concurrency Level: 100
Time taken for tests: 1.008 seconds
Complete requests: 1000
Failed requests: 1
(Connect: 0, Receive: 0, Length: 1, Exceptions: 0)
Write errors: 0
Non-2xx responses: 1
Total transferred: 253875 bytes
HTML transferred: 11936 bytes
Requests per second: 992.32 [#/sec] (mean)
Time per request: 100.773 [ms] (mean)
Time per request: 1.008 [ms] (mean, across all concurrent requests)
Transfer rate: 246.02 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 1 1.7 0 13
Processing: 10 79 18.8 80 122
Waiting: 9 79 18.8 79 122
Total: 10 80 18.9 80 122
Percentage of the requests served within a certain time (ms)
50% 80
66% 88
75% 94
80% 98
90% 102
95% 108
98% 113
99% 114
100% 122 (longest request)Rails 2.3.2 / JRuby 1.3-dev build 6586 (Single-Threaded)
Platform: JDK7 B56
# ab -n1000 -c100 http://somedomain:3000/main/index This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking somedomain (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: Server Hostname: somedomain Server Port: 3000 Document Path: /main/index Document Length: 11 bytes Concurrency Level: 100 Time taken for tests: 3.208 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 253000 bytes HTML transferred: 11000 bytes Requests per second: 311.71 [#/sec] (mean) Time per request: 320.810 [ms] (mean) Time per request: 3.208 [ms] (mean, across all concurrent requests) Transfer rate: 77.01 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 1 2.5 0 31 Processing: 37 304 56.4 318 350 Waiting: 36 304 56.5 318 349 Total: 37 305 56.5 318 352 Percentage of the requests served within a certain time (ms) 50% 318 66% 326 75% 330 80% 332 90% 336 95% 341 98% 345 99% 348 100% 352 (longest request)
Rails 2.3.2 / Ruby 1.9.1
Compiled: GCC -O3 -fomit-frame-pointer (SunCC failed to compile)
# ab -n1000 -c100 http://somedomain:3000/main/index This is ApacheBench, Version 2.3 <$Revision: 655654 $> Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ Licensed to The Apache Software Foundation, http://www.apache.org/ Benchmarking somedomain (be patient) Completed 100 requests Completed 200 requests Completed 300 requests Completed 400 requests Completed 500 requests Completed 600 requests Completed 700 requests Completed 800 requests Completed 900 requests Completed 1000 requests Finished 1000 requests Server Software: thin Server Hostname: somedomain Server Port: 3000 Document Path: /main/index Document Length: 11 bytes Concurrency Level: 100 Time taken for tests: 1.656 seconds Complete requests: 1000 Failed requests: 0 Write errors: 0 Total transferred: 267001 bytes HTML transferred: 11000 bytes Requests per second: 603.92 [#/sec] (mean) Time per request: 165.585 [ms] (mean) Time per request: 1.656 [ms] (mean, across all concurrent requests) Transfer rate: 157.47 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 0.6 0 8 Processing: 28 160 39.0 187 222 Waiting: 13 145 38.5 143 209 Total: 28 160 39.0 187 222 Percentage of the requests served within a certain time (ms) 50% 187 66% 190 75% 191 80% 191 90% 196 95% 201 98% 221 99% 221 100% 222 (longest request)
Conclusion
Seems like Catalyst has the edge in controller performance compared to Rails on MRI Ruby 1.8.7. Catalyst’s controller processing is 135% faster than Rails in single process performance and 471% faster as a forking multi process. It is nice to see that the Catalyst team addressed the controller performance short comings of the earlier versions of Catalyst. Like any benchmark take it with a grain of salt. In a real application your data access layer will most likely be the bottle neck.
Rails 2.3.2 under JRuby with threading enabled ran 283% faster than with MRI Ruby 1.8.7. I am anxiously waiting on JDK7 B57 with invoke dynamic support, this should help push JRuby’s performance even further. I guess I know what deployment option I will choose when deploying Rails.
Pick your poison, both frameworks provide excellent controller response performance. Keep in mind scaling is all about architecture and not how fast your controller’s responses are. That said, having an efficient framework does help 
Finding Circular Reference leaks in Perl
5 Comments | Filed under programming (4 votes, average: 3.75 out of 5) Loading ...Dynamic type languages such as Perl, Ruby, PHP, and Python free you as the developer from managing memory in your application. However, it isn’t a fool proof solution that you won’t have memory leaks in your application. You as the developer should be aware of how the underlying garbage collector of your preferred language works to accommodate for the inadequacies of its garbage collection algorithm.
Currently there are two ways of doing garbage collection; mark and sweep and reference counting. The Perl interpreter uses the latter. Reference counting is a fairly simple garbage collection technique. Each time you declare an instance, the reference count increments by one. When your program reaches the end of scope, objects with a reference of one get collected. However, if your object has a reference count of two it is kept. The one main draw back of reference counting is the fact it can’t deal with circular references. This is when both objects point to each other and they never get garbage collected.
On the other hand, Ruby and Java use the mark and sweep garbage collector. I personally have mixed feelings about it, since I don’t know exactly when my objects will be collected. The way mark and sweep garbage collection works, is it does not collect anything for a period of time. At intervals when the heap gets full, it runs its garbage collection. The downside to this is you don’t know exactly know when this happens and if there are lots of objects to be collected this leads to “stutters” and unresponsiveness of the application. If you have ever used a Java swing application you might have noticed these stutters, this is when garbage collection is taking place. However, it’s not as gloomy as I set the pretense to be with the mark and sweep garbage collection. Mark and sweep garbage collection can handle cyclic references unlike with reference counting, which is a huge boon to its usefulness. There has been much work done on mark and sweep garbage collection, specifically with generational mark and sweep collectors that try to fix the unresponsiveness issue. Java currently uses a generation GC, and Ruby hopes to obtain a generational GC for the Ruby 2.0 interpreter. Ideally a generational garbage collector would be the preferred GC for a long-standing process.
With that little garbage collection background out of the way, lets look at the life cycle of a instance in reference counting garbage collector.
Here is an example of how reference counting works ideally:
my $i = 5;
$i + 5;
print $i . ‘n‘;
} # $i should be garbage collected when it goes out of scope.
Unlike mark and sweep garbage collection with reference counting, you know exactly when your instance gets collected.
Here is a very simple problematic case for reference counting:
my $a;
my $b;
$a->{b} = $b;
$b->{a} = $a;
} # since both are pointing to each other they will never get collected.
This is a fairly simple case of where reference counting falls right on its face. Usually this isn’t a problem since most Perl scripting revolves around short-lived scripts. However, with frameworks such as Catalyst that are long running perl scripts this becomes an issue quickly. Thankfully, with Perl it is extremely easily to nail memory leaks, more so than with Ruby or Java. Enter Devel::Cycle and Devel::Peek, both of these modules can be installed from cpan. Both Devel::Cycle and Devel::Peek can assist you in tracking down the memory leak in a relatively short time.
use Devel::Peek;
foreach (1) {
my $parent = {name => ‘victor’ };
my $child = {name => ‘victor jr’ };
$parent->{child} = $child;
$child->{parent} = $parent;
find_cycle($parent);
# find_cycle belongs to Devel::Cycle
# which prints out the
# circular reference to STDOUT
Dump($parent);
# Dump belongs to Devel::Peek , its extra verbose
# which prints out the reference count to STDOUT
}
# Sample output # ibook:~/Desktop victori$ perl blah.pl # Cycle (1): <-- find_cycle tells you literly where the cyclic reference leak is at. # $A->{'child'} => %B # $B->{'parent'} => %A # # SV = RV(0x1817898) at 0x1800ec8 # REFCNT = 1 # FLAGS = (PADBUSY,PADMY,ROK) # RV = 0x18006dc # SV = PVHV(0x1830980) at 0x18006dc # REFCNT = 2 <-- Notice the reference count of 2 , we know we have a leak # FLAGS = (SHAREKEYS) # IV = 2 # NV = 0 # ARRAY = 0x404e60 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "name" HASH = 0xe6e17f14 # SV = PV(0x1801460) at 0x1800ea4 # REFCNT = 1 # FLAGS = (POK,pPOK) # PV = 0x401730 "victor" # CUR = 6 # LEN = 8 # Elt "child" HASH = 0x33ec6b5 # SV = RV(0x1817870) at 0x1832ca4 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x1800484 # SV = PVHV(0x18309b0) at 0x1800484 # REFCNT = 2 # FLAGS = (SHAREKEYS) # IV = 2 # NV = 0 # ARRAY = 0x404db0 (0:6, 1:2) # hash quality = 125.0% # KEYS = 2 # FILL = 2 # MAX = 7 # RITER = -1 # EITER = 0x0 # Elt "parent" HASH = 0xa99c4651 # SV = RV(0x18178a0) at 0x1832c44 # REFCNT = 1 # FLAGS = (ROK) # RV = 0x18006dc
So how do we fix this? Quite simple, all we do is weaken the reference count using weaken(). Here is a proper way of patching up the memory leak we introduced in our program.
use Devel::Peek;
use Scalar::Util qw/weaken/;
foreach (1) {
my $parent = {name => ‘victor’ };
my $child = {name => ‘victor jr’ };
weaken($parent->{child} = $child);
# we weaken the reference at the parent and all is well.
$child->{parent} = $parent;
find_cycle($parent);
# find_cycle belongs to Devel::Cycle which prints out the
# circular reference to STDOUT
Dump($parent);
# Dump belongs to Devel::Peek , its extra verbose
# which prints out the reference count to STDOUT
}
We weaken the reference at the parent level to set it back to a reference count of 1, so when it reaches the end of scope it will be collected and the memory leak will be no more.
Hopefully this is a good primer for other Perl coders out there who are facing memory leaks in their running long running perl scripts.
