Comments on: How to detect which language a text is written in? Or when science meets human! http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/ Sun, 22 Nov 2009 23:35:16 +0000 http://wordpress.com/ hourly 1 By: freelancer http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1819 freelancer Mon, 27 Apr 2009 14:29:57 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1819 This n-gram based tool written in PHP detects a site language depending on its content (for a user specified URL) http://www.site-language.com Sometimes have problems with non utf-8 encodings, but mostly works ok. This n-gram based tool written in PHP detects a site language depending on its content (for a user specified URL)

http://www.site-language.com

Sometimes have problems with non utf-8 encodings, but mostly works ok.

]]> By: feedbackmine http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1815 feedbackmine Thu, 09 Apr 2009 05:17:52 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1815 Here is my n-gram based language detector, written in ruby: http://github.com/feedbackmine/language_detector/tree/master I have been using it in production and it performs very well. Here is my n-gram based language detector, written in ruby:

http://github.com/feedbackmine/language_detector/tree/master

I have been using it in production and it performs very well.

]]> By: Check string language | keyongtech http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1806 Check string language | keyongtech Sun, 18 Jan 2009 17:07:58 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1806 [...] Check string language Also : http://tnlessone.wordpress.com/2007/...e-meets-human/ and more generally : http://www.google.fr/search?hl=fr&q=...+natural&meta= -- Patrice [...] [...] Check string language Also : http://tnlessone.wordpress.com/2007/…e-meets-human/ and more generally : http://www.google.fr/search?hl=fr&q=…+natural&meta= — Patrice [...]

]]> By: Neotenic http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1792 Neotenic Fri, 11 Jul 2008 23:07:01 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1792 I have made a language detector. It’s available here: <a href="http://www.anticulture.net" rel="nofollow">Language Detector</a> I have made a language detector. It’s available here:
Language Detector

]]> By: Neotenic http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1791 Neotenic Fri, 11 Jul 2008 23:03:39 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1791 I have made a language detector. It's available here: <a href="http://www.anticulture.net/LanguageDetector.php" rel="nofollow">Language Detector</a> I have made a language detector. It’s available here:
Language Detector

]]> By: My daily readings 03/06/2008 « Strange Kite http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1771 My daily readings 03/06/2008 « Strange Kite Thu, 06 Mar 2008 11:50:14 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1771 [...] How to detect which language a text is written in? Or when science meets human! « The Nameless One [...] [...] How to detect which language a text is written in? Or when science meets human! « The Nameless One [...]

]]> By: test 03/05/2008 « Strange Kite http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1770 test 03/05/2008 « Strange Kite Wed, 05 Mar 2008 17:50:16 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1770 [...] How to detect which language a text is written in? Or when science meets human! « The Nameless One [...] [...] How to detect which language a text is written in? Or when science meets human! « The Nameless One [...]

]]> By: Morticia http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1727 Morticia Tue, 27 Nov 2007 23:04:00 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-1727 I bought a scarf the other day and it has some writing on it. I'm really curious as to what it says because I Don't want to wear it if its some sort of satanic rambling. By looking at the print on it you'd say it was turkish or iranian or sopmething but I tried to translate from a few languages and still couldn't find out what it means. How can I go about finding out the meaning of this phrase? By the way the writing reads 'dariguguseman tujuhjemaatyalomsawah l' I bought a scarf the other day and it has some writing on it. I’m really curious as to what it says because I Don’t want to wear it if its some sort of satanic rambling. By looking at the print on it you’d say it was turkish or iranian or sopmething but I tried to translate from a few languages and still couldn’t find out what it means.
How can I go about finding out the meaning of this phrase?

By the way the writing reads ‘dariguguseman tujuhjemaatyalomsawah l’

]]> By: tnlessone http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-169 tnlessone Wed, 16 May 2007 08:18:56 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-169 Hi reto! Thanks for sharing your experience! It's always interesting to see the domains language detection can be applied to and the solutions chosen. And sorry for the comment blocking... It seems as if Askimet is very restrictive at... spam-freeing comments. It even blocks mine... Hi reto!
Thanks for sharing your experience! It’s always interesting to see the domains language detection can be applied to and the solutions chosen.
And sorry for the comment blocking… It seems as if Askimet is very restrictive at… spam-freeing comments.
It even blocks mine…

]]> By: reto http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-165 reto Tue, 15 May 2007 23:31:53 +0000 http://tnlessone.wordpress.com/2007/05/13/how-to-detect-which-language-a-text-is-written-in-or-when-science-meets-human/#comment-165 Your comments are very much appreciated! I myself started to ponder over language-detection using n-grams, when I wondered how DivX subtitle that can be uploaded to http://www.opensubtitles.org/en are detected and how the range of detectable languages could be increased. I found a PERL version by Gertjan van Noord's TextCat http://www.let.rug.nl/~vannoord/TextCat/ and a PYTHON version by Thomas Mangin. http://thomas.mangin.me.uk/software/python.html but I had difficulties to encounter a PHP version...and that's why I did it myself at http://boxoffice.ch/pseudo/index.php Since I'm definitely not experienced in writing any code I was surprised that three small functions together with the finger-prints of the languages do the job. There was no big issue about Asian (unicode) language-detection, since the php string funtiction go obiously bytewise (unless not told differently). I put the code, demo, finger-prints and more on my website as well. It might be instructive for some. Your comments are very much appreciated!
I myself started to ponder over language-detection using n-grams, when I wondered how DivX subtitle that can be uploaded to

http://www.opensubtitles.org/en

are detected and how the range of detectable languages could be increased.

I found a PERL version by Gertjan van Noord’s TextCat
http://www.let.rug.nl/~vannoord/TextCat/

and a PYTHON version by Thomas Mangin.
http://thomas.mangin.me.uk/software/python.html

but I had difficulties to encounter a PHP version…and that’s why I did it myself at
http://boxoffice.ch/pseudo/index.php

Since I’m definitely not experienced in writing any code I was surprised that three small functions together with the finger-prints of the languages do the job. There was no big issue about Asian (unicode) language-detection, since the php string funtiction go obiously bytewise (unless not told differently).

I put the code, demo, finger-prints and more on my website as well. It might be instructive for some.

]]>