Skip to content

IMPLEMENTATION_COMPARISON.md

Latest commit

 

History

History
228 lines (187 loc) · 9.21 KB

IMPLEMENTATION_COMPARISON.md

File metadata and controls

228 lines (187 loc) · 9.21 KB
Raw

Featured Groups Implementation Comparison

❌ Old Implementation (Unreliable)

def get_custom_featured_groups(count: int = 1):
    # Get all groups with basic fields
    groups = toolkit.get_action('group_list')(
        {},
        {'all_fields': True, 'include_extras': True}
    )
    
    # Filter for featured groups
    # ❌ PROBLEM: is_featured field not always returned by group_list!
    featured_group_ids = [
        group.get('name') for group in groups 
        if group.get('is_featured')  # Sometimes None, sometimes missing
    ]
    
    # Get full details
    groups_data = []
    for group_id in featured_group_ids:
        if len(groups_data) >= count:
            break
        group = toolkit.get_action('group_show')(context, data_dict)
        groups_data.append(group)
    
    return groups_data

Problems:

  • group_list() with include_extras=True doesn't reliably return scheming fields
  • Scheming stores custom fields in group_extra table
  • CKAN's list actions don't always join with extras
  • Results in inconsistent behavior ("sometimes it works, sometimes it doesn't")

✅ New Implementation (Reliable)

Step 1: Custom Action (queries database directly)

@tk.side_effect_free
def featured_group_list(context, data_dict):
    """Query database directly for featured groups."""
    
    # ✓ Direct database query - always finds the field
    query = model.Session.query(model.Group.name).join(
        model.GroupExtra,
        model.Group.id == model.GroupExtra.group_id
    ).filter(
        model.Group.is_organization == False,
        model.Group.state == 'active',
        model.GroupExtra.key == 'is_featured',
        model.GroupExtra.value.in_(['True', 'true', '1', 'yes'])
    ).distinct()
    
    return [name for name, in query.all()]

Step 2: Core CKAN Pattern Helper

def featured_group_org(items, get_action, list_action, count):
    """Core CKAN pattern from ckan/lib/helpers.py"""
    
    def get_group(id):
        context = {
            'ignore_auth': True,
            'limits': {'packages': 2},
            'for_view': True
        }
        data_dict = {
            'id': id,
            'include_datasets': True
        }
        try:
            return logic.get_action(get_action)(context, data_dict)
        except logic.NotFound:
            return None
    
    groups_data = []
    extras = logic.get_action(list_action)({}, {})
    
    found = []
    # ✓ Iterate through featured items first, then extras
    for group_name in items + extras:
        group = get_group(group_name)
        if not group or group['id'] in found:
            continue
        found.append(group['id'])
        groups_data.append(group)
        if len(groups_data) == count:
            break
    
    return groups_data


def get_custom_featured_groups(count=1):
    """Public helper function."""
    
    # ✓ Get featured names from reliable custom action
    featured_names = toolkit.get_action('featured_group_list')({}, {})
    
    # ✓ Use core pattern to get full details
    return featured_group_org(
        items=featured_names,
        get_action='group_show',
        list_action='group_list',
        count=count
    )

Benefits:

  • ✅ Reliable - Database query always finds the field
  • ✅ Consistent - Works the same way every time
  • ✅ Complete - Returns full details like core CKAN
  • ✅ Efficient - Only queries full details for items that will be returned
  • ✅ Follows core pattern - Matches CKAN's own implementation

Core CKAN vs Your Implementation

Aspect Core CKAN Your New Implementation
Data Source Config file (ckan.featured_orgs) Database schema field (is_featured)
Management Edit config, restart CKAN Edit in UI, immediate effect
Pattern featured_group_org() Same! featured_group_org()
Fallback All groups/orgs from list action Same! All groups/orgs from list action
Output Full details with datasets Same! Full details with datasets
Reliability ✅ Always works ✅ Now always works!

Data Flow Diagram

┌─────────────────────────────────────────────────────────────────┐
│ Template calls: h.get_custom_featured_groups(count=3)           │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ Helper: get_custom_featured_groups()                            │
│   1. Call featured_group_list() action                          │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ Action: featured_group_list()                                   │
│   2. Query database: SELECT name FROM group                     │
│      JOIN group_extra WHERE key='is_featured' AND value='True'  │
│   3. Returns: ['group1', 'group2']                              │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ Helper: featured_group_org(['group1', 'group2'], ...)           │
│   4. For each featured group:                                   │
│      - Call group_show() to get full details                    │
│      - Include datasets (limited to 2)                          │
│   5. If count not reached, add more from group_list()           │
│   6. Returns: [                                                 │
│        {name, title, description, packages: [...], ...},        │
│        {name, title, description, packages: [...], ...},        │
│        {name, title, description, packages: [...], ...}         │
│      ]                                                           │
└────────────────────────────┬────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────┐
│ Template receives: 3 groups with full details                   │
│ {% for group in featured_groups %}                              │
│   <h3>{{ group.title }}</h3>                                    │
│   {% for package in group.packages %}...{% endfor %}            │
│ {% endfor %}                                                     │
└─────────────────────────────────────────────────────────────────┘

Why the Old Code Failed Intermittently

The group_list() action:

# Sometimes returned:
[
  {'id': '123', 'name': 'group1', 'is_featured': True},  # ✅ Works!
  {'id': '456', 'name': 'group2'}  # ❌ Missing is_featured!
]

# Other times returned:
[
  {'id': '123', 'name': 'group1'},  # ❌ No extras at all!
  {'id': '456', 'name': 'group2'}
]

Why? CKAN's list actions are optimized for performance and don't always load extras, especially for scheming fields. The behavior depends on:

  • CKAN version
  • Which plugins are loaded
  • How the action is called
  • Database query optimization

Solution? Query the database directly in a custom action!

Migration Checklist

  • ✅ Custom actions added (featured_group_list, featured_organization_list)
  • ✅ Helpers updated to use core CKAN pattern
  • ✅ Database query handles multiple boolean formats
  • ✅ Tests updated
  • ✅ Documentation created
  • 🔄 Test on your CKAN instance
  • 🔄 Verify featured items are returned
  • 🔄 Run debug script if issues persist

Next Steps

  1. Restart CKAN to load new code
  2. Test with debug script to see what's in database
  3. Check featured items are displayed correctly
  4. Report any issues with specific error messages