Archive for the ‘PyGuile’ Category

PyGuile - Part 5 - Python objects (PyObjects) as proxies for Guile objects (SCMs)

Thursday, October 9th, 2008

An essential part of integration of Scheme (as implemented in Guile) and Python is allowing Python code to call back code implemented in Scheme. It is also desirable to be able to access data and invoke methods on otherwise-opaque objects created and managed in Scheme. The specifics of opaque object access should also be independent of the specific object system being used in the Scheme-written part of the application.

When implementing in PyGuile the proxying scheme, in which PyObjects serve as proxies for SCMs, the following need to be taken into account:

  • Data type conversion
  • Retention of references as a protection against garbage collection

Data type conversions

A Python callback can be expected to receive positional and keyword arguments, and return a result of any type. Therefore, templates (possibly trivial) for converting between PyObjects (Python data types) and SCMs (Guile data types) need to be associated with each callback.

In the case of objects, we need to associate, with each attribute getter, a template for converting the value from SCM into PyObject. With each attribute setter, a template for converting the value from PyObject into SCM needs to be associated. With each method, which can be invoked on the object, minimum two templates are needed. Three templates should be provided, in case the object needs to be manipulated by an interface, which expects both positional and keyword arguments in the object’s methods.

All the templates needed to work with a SCM (as a callback or as an object) are associated with it when it is wrapped as it is being passed from Guile to Python.

Retention of references

PyObjects, which wrap SCMs, are not expected to be seen by Guile’s garbage collector. Therefore, we need a mechanism for protecting SCMs referenced by PyObjects.

Due to efficiency considrations, Guile’s scm_permanent_object, scm_gc_protect_object or scm_gc_unprotect_object should not be used on every SCM passed to Python. The solution is to create a set object in Guile, protect it using scm_permanent_object (a single call) and then register in it all wrapped SCMs. When a wrapping PyObject’s __del__ function is invoked, one of its actions is to remove the corresponding SCM object from the set. The set will be implemented using a standard hash table, whose keys will be indexes and the data - the SCMs themselves.

PyGuile - version 0.1.0 has been released for beta testing!

Wednesday, September 24th, 2008

You’ll find it in
DISCLAIMER: the software is still at beta level, and has never been tested outside of my own PC.

UPDATE: At  22 May 2016, I uploaded the project to Github. You may find it in

PyGuile - Part 4 - Argument and result conversion issues

Monday, September 22nd, 2008

There is no 1:1 mapping between Scheme and Python data types. As a consequence, there are several cases, in which PyGuile has to guess how would the user like to have the arguments and result of a Python function converted. Instead of guessing, we would like to empower the user to be explicit about the kinds of conversion which he wants.

The following is a census of ambiguous data conversion cases, which I identified.

  1. Scheme pair ->
    • Python 2-Tuple
    • Python 2-List
  2. Scheme list ->
    • Python Tuple
    • Python List
    • Nested tree of pairs (2-Tuples or 2-Lists)
  3. Python 2-Tuple or 2-List ->
    • Scheme pair
    • Scheme list
    • Scheme rational (if the Pythonic data structure consists of two integer values)
  4. Scheme alist (association list) ->
    • Python Dict
    • Python Tuple/List of 2-Tuples
  5. Python string ->
    • Scheme string
    • Scheme symbol
    • Scheme keyword

    Additional considrations:

    • Case sensitivity of symbols and keywords
    • String representation of keywords in Guile has leading dash - to retain or remove it in the Python side of affairs?
  6. Python 1-character string ->
    • Scheme char
    • Scheme string

    Additional considration: utf-8 encoded glyph is a sequence of few characters.

  7. Python int ->
    • Scheme int
    • Scheme bignum
    • Scheme char
  8. Python None -> One of several possible values: ‘(), #f, SCM_EOL, ‘*None* or another custom Scheme value.
  9. Python (),[],{} ->
    • Scheme ‘()
    • SCM_EOL
    • Custom Scheme value
  10. Scheme ‘() ->
    • Python ()
    • Python []
    • Python {}
  11. SCM_EOL ->
    • Python (),[],{}
    • Python None
    • Custom Python value
  12. Scheme rational ->
    • Python Float
    • 2-Tuple of Python Ints
  13. Scheme exact/inexact flag in numerical values - if and how to represent it in the Python side of the application?
  14. Giant data structures with sparse access needs - lazy vs. eager conversion
  15. Exception objects
  16. Objects of certain classes (vectors, ports, functions, images, etc.)

There is also the separate issue of string encoding/decoding, with which we deal by mandating that anything passing between Scheme and Python has to be utf-8 encoded.

One of the goals of PyGuile is to make it efficient to invoke Python library functions from Guile. Therefore, efficiency of conversion of function arguments and results is critical.

When there are no user hints, the following inefficiencies occur:

  1. PyGuile has to make a default (and possibly sub-optimal) choice when encountering one of the above ambiguous cases. Then the script using the data has to reformat it to match the data format to its actual needs.
  2. PyGuile has to identify the data type of each datum. The present implementation does not go into the internal representation of Guile (SCM) and Python (PyObject) objects, therefore PyGuile has to test for various data types one by one, until one of them matches the argument.
  3. Sometimes a Python procedure needs to do no processing on one of its arguments. The argument’s value needs only to be passed around as a pointer, or to be inserted into the right place in a result data structure. In such a case, it is desirable to use the most efficient conversion possible i.e. wrap/unwrap opaque objects. This is a generalization of the case of giant data structures with sparse access needs.

Therefore, when performance is critical, hints from the user would help not only to disembiguate the conversion process but also to speed it up.

The user hints will be implemented as follows.
With each function (Python function invoked from Guile, or Guile function invoked from Python) we associate two (possibly degenerate) signatures. One signature will contain the hints for converting the function’s arguments. The second signature will hint how to convert the function’s result. The signatures are Scheme lists, whose leaf nodes are symbols denoting conversion functions.

Chris Jester-Young, in his answer to my question in Stackoverflow, proposed the following function for traversing two corresponding tree structures, and applying the functions in one of them to data in the other one.

  (define (map-traversing func data)

    (if (list? func)

        (map map-traversing func data)

        (func data)))

Using it requires unquoting. Example:

  (map-traversing `((,car ,cdr) ,cadr) '(((aa . ab) (bb . bc)) (cc cd . ce)))

Our implementation will differ from the above in details, as the signatures’ leaf nodes do not denote proper Scheme functions.

PyGuile - Part 3 - Non-goals

Sunday, September 21st, 2008

Every project, including software development projects, needs an identity. It needs a definition of its boundaries. It has to be clear about what is inside it and what is outside of it.

Without such a definition, the project would try to be too many things for many people, and as a result, its products would not be really useful for anyone.

A project’s identity makes it easier and faster to make design and trade-off decisions.

Given the above trite introduction, and given that there is a list of goals for the PyGuile project, a list of non-goals is needed as well and here it is.

  1. Theoretical academic purity - attempt to convert every data type from Guile to Python and vice versa, and to support the whole range of values assumed by each data type.
  2. Ability to mix code snippets from both Scheme and Python in the same source code file.
  3. Invocation of machine language libraries (static or DLLs) - for this purpose, there are already existing tools (SWIG and PerlXS).
  4. Framework for making it easy to add support for interoperation with yet another scripting language.

There are also some goals, which are low priority and I do not plan to shed tears if they prove to be impossible to achieve without significant effort:

  1. Thread-safety
  2. Tail recursion support

PyGuile - Part 2 - Design Issues

Saturday, September 20th, 2008

While working on the PyGuile, I identified the following design issues.

  1. The data type trees of Scheme and Python do not have an 1:1 correspondence.
    • Do we want to convert a Scheme list into a Python Tuple or a Python List?
    • How about an alist (associative list) - should be a Python List of 2-tuples or a Python Dict?
    • And in the other direction - do we want to convert a Python string into a Scheme string, symbol or keyword?
  2. API for adding plugins which convert between Guile and Python representations of useful data types (such as file handles, images or Berkeley sockets).
  3. How do we want to pass large data structures - convert them immediately, or employ lazy conversion (convert an element only when it is requested)? If we employ lazy conversion, how do we implement the associated bookkeeping? See more about this below.
  4. How do we deal with the different garbage collection regimes of Guile and Python? In particular, how do we make SCM objects owned by Python objects known to the Guile garbage collector?
  5. How will we support Unicode? Bear in mind that we want to minimize manipulations of long text strings.
  6. How to allow each scripting language to seamlessly invoke functions in the other scripting language?

The problem of lack of 1:1 correspondence will be dealt with as follows.

A standard conversion convention, which will work for the overwhelming majority of cases, will be employed. Functions, which have special needs, will have their argument conversions specified by means of a suitable tree-structured template.

When passing a data structure (or object) created in language A to language B, the following cases can happen:

  1. Opaque pointer - B only passes it around. A performs all processing and B just holds the pointer for future reference.
  2. B accesses a single element (or small number of elements) in the data structure.
  3. B loops over all elements of the data structure.
  4. B needs arbitrary access to several elements of the data structure (example: image processing).

Those cases can be dealt with as follows:

  • Case 1 can be handled by wrapping a language A pointer by a language B object, which carries opaque data around.
  • Cases 2,3 can be dealt by means of custom data access procedures (such as Python’s __getitem__()). An element will be converted only when it is actually requested. Elements in nested data structures can be dealt with as in case 1.
  • Case 4 can be handled by implementing a mechanism for plugging in and registering custom conversion functions for specific data types.

In practice, the most tough design issue, which I identified so far, is the management of the SCM objects owned by Python objects.

When a SCM object is assigned to an attribute of a Python object, some registration mechanism needs to
be invoked so that the SCM object can be reclaimed by the Guile garbage collector if the Python object goes out of scope. The registration mechanism needs also to take care of marking the SCM objects while they are owned by a living Python object.

PyGuile - Part 1 - Using Python libraries in Guile (a Scheme implementation) scripts

Friday, September 19th, 2008

For long time I have dreamt of invoking Python libraries from scripts written in Scheme. The reason for this is to be able to enjoy the fantastically rich control structures possible in Scheme, yet use familiar libraries to accomplish useful actions, some of which are unavailable in SLIB and other Scheme libraries.

Now at last I am working on realizing this dream. The Scheme implementation being used is version 1.6 of Guile and the Guile extension being developed embeds a Python 2.4 interpreter. In the future, more recent versions of Guile and Python will be used.

The goals of the project are:

  1. Make it easy to invoke Python libraries from Guile.
  2. The integration between Python and Guile is to be seamless.
  3. The architecture of the implementation shall enable optimizations for efficient runtime behavior.

To accomplish those goals, it is necessary to:

  1. Convert primitive Scheme data types (integers, reals, Booleans, strings, lists) into the corresponding Python data types, and vice versa.
  2. Be able to invoke functions defined in one language from the other language. This has to be bidirectional in order to support callbacks.
  3. Be able to pass around pointers to objects (as opaque values) and invoke methods over them.
  4. Have efficient transfer of control and data between both languages.
  5. Deal with different garbage collection conventions in both environments.
  6. Be able to optimize code for a particular pair of language runtime systems.
  7. Nice to have: support for recursion, especially tail recursion.
  8. Nice to have: thread-safety.

It is envisioned that the software developed in this project will be part of a larger system, which will allow more scripting languages to interoperate with Guile and with each other.

There is another project - Schemepy - which embeds a Scheme interpreter in Python scripts.  This project has different focus and it essentially allows Scheme to be used for those parts of a project, in which its strengths are especially important.

mirapex discount generic cheap vytorin in usa cheap kamagra london cheap glucotrol xl it works buy baytril injectable cheap actoplus met side effects buy tulasi plant online bangalore baclofen discount code buy bupropion sr quit smoking buy hydrea you buy anti fungal dog shampoo brand cialis discount name buy online order alphagan notice xenical discount vouchers buy aricept europe abana discount vouchers cheap glucotrol xl prescribing information cheap viagra jelly fast delivery order adalat oros buy aciclovir tablets over the counter cheap levitra professional who makes buy dilantin angeles order hytrin buy prednisone mexico cheap wellbutrin sr dosage for weight loss buy men;s health ipad buy clean perfume cheap advair diskus rinse mouth after use lipothin discount code buy viagra professional vs viagra cheap toprol xl ingredients order requip uk buy trazodone online wellbutrin sr discount experiences buy depo medrol for dogs order zanaflex dosage order imuran india celebrex cost in usa buy mentat forum cheap brand viagra the uk order alli zwaneven cheap malegra dxt quiver buy zithromax mexico benicar savings plan order v gel za bolec(ine v mis(icah order tadacip tablete cheap glucotrol xl viagra soft discount card order female viagra wiki cheap lisinopril version buy capoten pills cheap apcalis sx tadalafil 20 mg buy cardizem where can order trazodone xanax order furosemide mg cheap vasotec uk mentax discounts buy kamagra oral jelly romania buy propranolol online ireland buy brand viagra from canada viagra soft discount generic cheap muscle relaxant for neck pain cheap shuddha guggulu commiphora wightii buy premarin cream uk buy cytotec united states allegra discount generic buy desyrel you order singulair samples cheap vytorin a prescription buy alavert miami flomax order canada buy innopran xl for anxiety cheap cholesterol lowering drugs celebrex discount card 2014 cheap suprax cw cheap levitra professional paypal best cheap hair loss treatment ranitidine discount buy tinidazole buy asacol cheap cheap motrin suspension order vantin balma buy female cialis and viagra cheap penis enlargement buy hydrea real buy lamisil vegas cheap suhagra force buy xenical vallejo order v gel by himalaya where to buy propecia yahoo buy isoniazid usa cheap benicar bula cheap v gel south beach buy avodart in canada buy olanzapine jubilant buy cholesterol lowering home remedies buy clomid australia buy lasix injection pyridium discount vouchers can you buy unisom over the counter propranolol discount vouchers order imuran back buy lioresal mg aricept discount cheap allegra d 24 hour order mentax prescription buy generic lamictal online liv 52 discount for sale buy tamoxifen uk order rhinocort vidal buy cleocin cheap relafen uses cheap suhagra online ranitidine discount vouchers order terramycin dogs chloromycetin discount price cheap lamisil cheap diclofenac gel jonac prevacid discount voucher order tadalis sx order ginette 35 mrp cheap viagra jelly with dapoxetine buy asacol 400 mg retin a 0 05 discount janssen buy valtrex walmart buy men;s health zero excuses workout order urispas canada buy kamagra soft wirkung buy carafate japan buy colchicine usa buy xalatan order doxycycline xenia order benfotiamine glycation robaxin discount pharmacy cheap ginette 35 benefits order colchicine dosage buy olanzapine side discount on seroquel buy coumadin side order calcium carbonate or citrate buy flagyl in us order aricept a prescription buy men;s health issues cheap lady era quanto order baytril for dogs cheap v gel opi gel elocon to buy buy trental south africa cheap ciprofloxacin 500mg order antivert reviews buy clomid in australia buy cymbalta in mexico order terramycin hair loss discount experiences buy cyproheptadine for cats buy propecia online europe cheap hair loss treatment uk cheap cialis professional vs regular buy revia from uk buy cheap antabuse order diamox powder buy leukeran online uk order effexor xr forum trileptal discount card order neurontin buy propecia with prescription order anti fungal herbs for candida buy saw palmetto quercetin price combivent respimat order ginseng xiamen order abilify reviews cheap finpecia effectiveness order cialis professional acheter buy dipyridamole real buy propecia from boots buy alli-c garlic pills cleocin gel discount rosacea buy clonidine in stores cheap viagra and cialis buy aricept rx order brand cialis prescription cheap muscle relaxant gel india cheap caverta tablet cheap femara online cheap isoptin a prescription imuran discount online iodination of acetone order buy erythromycin uk buy rhinocort uk buy cialis tablets buy snovitra super power in italia plendil discount order minocycline australia order toprol xl juice buy levitra plus with mastercard buy haridra benefits where can i buy anti fungal shampoo buy strattera in uk alli discount loss tablets cheap femara a prescription apo discounter voltaren cheap cardura lek discount generic viagra buy menosan salvia buy cystone forum rocaltrol discount coupons buying ventolin inhalers order rosuvastatin in canada buy prandin legal order antabuse back cheap viagra jelly europe cheap diclofenac gel jonah complex order glucotrol xl pfizer order evista generic order zithromax for chlamydia buy protonix at walmart cheap diclofenac gel yellow cheap danazol nedir ventolin discount card cheap cefixime caverta discount vouchers order diclofenac gel pregnancy cheap ralista paskaleva cheap v gel jell lantus discount card cheap brand levitra dosage cheap prandin buy online order kamagra chewable soft cheap muscle relaxant valium eli lilly cymbalta discounts buy tinidazole vidal order rogaine 5 topical solu"syon cheap ranitidine in usa order calcium carbonate nitric acid cheap etodolac uses buy azulfidine mexico order metformin cheap toprol xl extended release buy shallaki ayurvedic buy accutane safe buy dipyridamole best place order ginette 35 hair loss order lexapro reviews plendil discount code chloromycetin discount copay how to buy penegra in india buy glucotrol xl maximum dose buy cholesterol lowering agents other than statins cheap nizagara information buy dutas australia order toradol a prescription cheap lopressor xl buy female cialis brand online buy celexa online cheap buy cleocin uk order apcalis sx preisvergleich cheap desyrel uses cheap effexor xr and weight gain buy alavert hong kong buy ortho tri cyclen kaiser buy propecia toronto elavil discount voucher order real cialis online buy coumadin zealand cheap cholesterol lowering products betnovate cream order buy erythromycin online buy augmentin 875 cheap lotensin dosage buy augmentin tablets sleep aid discount cvs order diclofenac japan order chloromycetin by mail order antabuse you order arrival rate cheap wellbutrin sr cut in half buy dipyridamole vancouver order cialis super active work order baytril for cats buy clonidine dosage buy hoodia patch pain relief discount quick cheap hair loss zinc supplements effexor xr discount half life buy estrace cream canada order metformin quickly cheap vasotec india order mango catalogue buy dostinex much inderal discount coupon buy arthritis aids cheap alli capsules buy cipro online shuddha guggulu discount malaysia buy cytotec in usa online buy desyrel at walmart order dried mango buy snovitra super power youtube order amantadine a prescription buy ciplox you cheap hair loss under arms order wellbutrin sr generic where to buy zofran in canada order rhinocort hayfever order relafen australia cheap penegra india buy viagra professional ingredients cheap muscle relaxant valerian discount for protonix buy toradol kuala lumpur buy zoloft brand online stromectol discount vouchers cheap female cialis pills order carafate you order lipitor side chloromycetin discount coupon order pain relief ear infection buy prandin angeles order alphagan bula cheap aristocort injection cheap neurontin side order naprosyn nedir buy singulair uk order rogaine 5 foam canada aldactone discount card buy rogaine 5 does it work order paxil back cheap abilify online order chloromycetin ointment buy propecia in thailand buy indomethacin uk order valtrex dosage cheap renovation tips cheap artane motels cheap claritin reditabs order erectile dysfunction pills online cheap risperdal australia order trial erection packs 1 buy retin-a 0 025 gel cheap flagyl er tablets cheap apcalis sx jelly review buy zyloprim india nitrofurantoin purchase canada buy arava japan buy januvia real avalide discount card cost celebrex walmart lynoral discount code buy desyrel real buy viagra plus glasgow buy casodex sales cheap diclofenac gel sodium cheap toprol xl label buy cialis kl buy confidor order promethazine with codeine online skelaxin discount vouchers buy nolvadex india order tamoxifen uk estrace discount vouchers buy eurax online cheap cafergot sale grifulvin v discount cvs buy orlistat hong kong buy clomid philippines cheap sumycin contraindications cheap plendil generic proscar to buy buy nolvadex bodybuilding cheap lady era lake fl viagra jelly discount generic buy aldactone in uk cheap provera questions cheap indinavir alternatives cheap suprax fi seroquel discount prices order erectile dysfunction reasons buy periactin line order cialis xertigny buy zyloprim in uk buy nolvadex uk bodybuilding cheap toprol xl vs atenolol cheap rental edmonton cheap elimite lice buy l tryptophan recommended dosage cheap renovation ideas buy ventolin online usa buy zithromax to treat chlamydia buy retino-a cream 0,025 buy robaxin uk order zantac jarabe buy quetiapine uk viagra professional discount review buy kamagra bangkok buy toradol uk cheap brand levitra professional order hair loss natural remedies order female viagra jacks kamagra gold discount ve'leme'nyek buy tricor real order lisinopril kidney buy lamictal buy tretinoin 0 05 refissa buy micronase online canada cheap cialis soft uk buy propecia germany men;s health discount zinc diclofenac gel discount xedenol environ skin care order online cheap viagra plus grapefruit juice order dipyridamole india buy clomid pct cheap purim ideas cheap cleocin buy buy house canal view lahore cheap viagra professional venta buy female viagra zealand female cialis discount effects cheap viagra plus usa order ginette 35 for pcos order hydrochlorothiazide a prescription order tretinoin 0 05 cream cheap diclofenac gel evaluation buy orlistat in uk buy tinidazole singapore order glycomet indinavir discount pharmacy order serevent prescription order tulsi seeds cheap rogaine 2 weeks buy men;s health look book cheap lariam online cheap vasotec get order glucophage india actonel discount program indinavir discount buy cheap zyvox order pain relief removal of wisdom teeth order diflucan rx buy isoniazid katg buy ginette 35 and acne cheap kamagra tablets cheap arjuna vs cheap zithromax xarope buy brand viagra in singapore buy quibron t sr dose cheap periactin online cheap cialis viagra online cheap hytrin ovulos buy levitra plus bangkok buy tenormin generic cheap dipyridamole buy buy lamisil generic cheap ortho tri cyclen emergency contraception lantus vial discount code buy female viagra boots buy accutane singapore order antabuse real order zyloprim mail cheap procardia headache buy zofran buy augmentin bangkok buy rosuvastatin in thailand cheap advair diskus klinefelter syndrome buy mircette ingredients buy premarin cream online uk buy triamterene can you buy prometrium online cheap lady era viagra buy terramycin you order levitra xanthi effexor xr generic cost buy cefixime liquid mango discount uk cheap cialis soft italia buy generic zovirax ointment buy abilify zealand cheap toradol australia hyaluronic acid discount yin yoga provera discount voucher buy xenical tablets uk buy viagra jelly glasgow buy second hand graphics card india buy tinidazole australia order zetia india order glyset canada order hair loss xeloda order zyloprim tabletas generic zoloft reviews buy rosuvastatin korea order tulasi plant online cardizem discount coupon order viagra sublingual spray buy glucotrol xl wikipedia order rogaine 5 vs liquid buy micronase pills cheap voveran sr quotes aleve coupon deals buy urispas legal order diflucan you